1
0
mirror of https://github.com/microsoft/qlib.git synced 2026-07-09 22:10:56 +08:00
Commit Graph

1135 Commits

Author SHA1 Message Date
Fivele-Li
7234308651 Add base config in yml (#1500)
* path on Windows contains double '/' which may cause open file failed.

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* add baseConfig in yml,user can add new keys or update/drop keys in baseConfig;

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* pip release version 23.1 on Apr.15 2023, CI failed to run, Please refer to #1495 ofr detailed logs. The pip version has been temporarily fixed to 23.0.1.

* 1.Search for baseConfig in multiple directories;
2.Add user instructions in qrun;

* fix format with black

* 1.modify baseConfig key to BASE_CONFIG_PATH;
2.only find config file in absolute path and relative path;

* load BASE_CONFIG_PATH on absolute path & relative path;

* fix Lint with black

---------

Co-authored-by: lijinhui <362237642@qq.com>
2023-05-12 17:35:37 +08:00
Chaoying
acf5df27ce Add support for redis password (#1508) 2023-05-08 16:17:15 +08:00
Chaoying
37a59f28d3 Fix deprecated syntax in numpy (#1507)
* Fix deprecated syntax in numpy

* Replace np.bool with bool
2023-05-08 16:17:02 +08:00
YQ Tsui
b084c352f5 provide dtype to empty series to surpress warning; fix type (#1449) 2023-05-05 17:47:44 +08:00
Maksim Zayakin
9e22e5168b Remove unused DNNModelPytorch params (#1470)
* Remove lr_decay and lr_decay_steps params

More flexible way to pass a scheduler (via callable function) is already
supported

* remove lr_decay and lr_decay_steps from mlp workflow configs
2023-04-28 17:48:40 +08:00
Huoran Li
7f1e8c5206 Refine Qlib RL data format (#1480)
* wip

* wip

* wip

* Fix naming errors

* Backtest test passed

* Why training stuck?

* Minor

* Refine train configs

* Use dummy in training

* Remove pickle_dataframe

* CI

* CI

* Add more strict condition to filter orders

* Pass test

* Add TODO in example

---------

Co-authored-by: Young <afe.young@gmail.com>
2023-04-26 21:14:30 +08:00
Fivele-Li
46264dfec9 normpath for Windows (#1495)
* path on Windows contains double '/' which may cause open file failed.

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* locate import numpy error

* pip release version 23.1 on Apr.15 2023, CI failed to run, Please refer to #1495 ofr detailed logs. The pip version has been temporarily fixed to 23.0.1.

---------

Co-authored-by: lijinhui <362237642@qq.com>
2023-04-26 16:26:12 +08:00
you-n-g
32c3070b73 Refine DDG-DA (#1472)
* Run ddg-da successfully

* Support include valid; More parameters

* Support L2 reg & visualization

* Blackformat

* Enable fill_method

* Support specify handler & optim dataset

* Fix Pylint
2023-04-07 15:00:21 +08:00
you-n-g
40de67265a Update Docs about some concepts in DataHandler (#1485) 2023-04-07 10:02:16 +08:00
BookSword
d010219ba6 Merge branch 'main' into bugfix 2023-03-23 16:11:19 +08:00
BookSword
4fc8a5f25f merge 2023-03-23 16:05:09 +08:00
Linlang
0e8bfcb5d3 fix_pylint_w0719 (#1463)
* fix_pylint_w0719

* remove_fixme
2023-03-17 19:25:49 +08:00
you-n-g
e457ca8511 Improve annotation & documentation for handler (#1312)
* Improve annotation & documentation for handler

* Add type
2023-03-15 21:15:40 +08:00
Huoran Li
653c082e7a Order execution open source (#1447)
* Waiting for bin data

* Complete readme

* CI

* Add inst filter by time

* Update qlib/data/dataset/processor.py

* typo

* Fix time filter bug

* Add Filter and set Universe

* Complete data pipeline

* Fix Provider Logger Info Args

* Add DQN; a minor bugfix in ppo reward.

* update readme. modify assertion logic in strategy check.

* Fix Doc issues and fix black

* Fix pylint Error

---------

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2023-03-13 12:06:28 +08:00
you-n-g
f98e04ca9d Fix Field Name Error 2023-03-03 16:28:47 +08:00
Huoran Li
5eb5ac1f1f RL backtest pipeline on 5-min data (#1417)
* Workflow runnable

* CI

* Slight changes to make the workflow runnable. The changes of handler/provider should be reverted before merging.

* Train experiment successful

* Refine handler & provider

* test passed

* Ready to test on server

* Minor

* Test passed

* TWAP training

* Add PPOReward

* Add a FIXME

* Refine PPO reward according to PR comments

* Minor

* Resolve PR comments

* CI issues

* CI issues

* CI issues
2023-02-13 12:43:22 +08:00
Young
6295939346 Update to Dev Version 2023-01-29 18:55:23 +08:00
Young
5f3e322784 Update Version 2023-01-29 18:53:25 +08:00
Huoran Li
d8fc9aea6b RL Training pipeline on 5-min data (#1415)
* Workflow runnable

* CI

* Slight changes to make the workflow runnable. The changes of handler/provider should be reverted before merging.

* Train experiment successful

* Refine handler & provider

* CI issues

* Resolve PR comments

* Resolve PR comments

* CI issues

* Fix test issue

* Black
2023-01-18 16:17:06 +08:00
YQ Tsui
d8764660dc [BUGFIX] allow sell in limit-up case and allow buy in limit-down case in topk strategy (#1407)
* 1) check limit_up/down should consider direction; 2) fix some typo, typehint etc

* fix error

* Update test_all_pipeline.py

Believe it's just some arbitrary number.
The excess return is expected to change when trading logic changes.

* add flag forbid_all_trade_at_limit to keep previous behivour for backward compatibility
2023-01-10 09:46:18 +08:00
Linlang
7f08e6c7b3 fix subprocess.check_output bug (#1409)
* fix_check_output_bug

* change_log_info

* recover_feature
2023-01-06 21:44:23 +08:00
YQ Tsui
4fe3ffccfd fix typo, staticmethod etc. (#1402)
* config.py: fix typo; static method

* fix typo in qlib/utils/paral

* 1) limit numpy version as numba support for 1.24+ has not been released; 2) no need to use custom numba version for pytest.

* remove useless argument

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2022-12-31 08:02:05 +08:00
YQ Tsui
2f5ce3dc01 Plot enhancement (#1390)
* horizontally put the bar figures

* 1) use rangebreaks to handle gaps in datetime axis instead of make them string; 2) allow simultaneously plot rankic in ic_figure

* pylint improvement

* fix black lint

* better axis formatting

* default not show gaps

* resolve doc built error

* fix pylint

* Update qlib/contrib/report/analysis_model/analysis_model_performance.py

More detailed description

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>

* Update qlib/contrib/report/analysis_model/analysis_model_performance.py

for Python backward compatibility

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>

* add doc string

* fix black

* 1) limit numpy version as numba support for 1.24+ has not been released; 2) no need to use custom numba version for pytest.

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2022-12-31 07:58:41 +08:00
Linlang
756bd0f65b Fix ZScoreNorm processor bug (#1398)
* fix_ZScoreNorm_bug

* fix_CI_error

* fix_CI_error

* add_test_processor

* fix_pylint_error

* fix_some_error_and_optimize_code

* modify_terrible_code

* optimize_code

* optimize_code
2022-12-30 20:42:37 +08:00
Chia-hung Tai
cbd69fb0ed The limit threshold in Taiwan stock market is also 10%. (#1391)
* The limit threshold in Taiwan stock market is also 10%.

* Warning limit_threshold when it is None.
2022-12-12 21:37:01 +08:00
YQ Tsui
5e3924d7a6 fix some typo in doc/comments (#1389)
* fix typo in docstrings

* fix typo

* fix typo

* fix black lint

* fix black lint
2022-12-11 14:29:16 +08:00
Young
26d24b5b23 Bump to Dev Version 2022-12-09 18:21:39 +08:00
Young
b8bb3adbc8 Bump Qlib Version 2022-12-09 18:06:11 +08:00
Chia-hung Tai
ea10da32ba Fix warning in processor.py. (#1386)
* Fix warning in processor.py.

* Remove comment.
2022-12-08 23:47:05 +08:00
Hyeongmin Moon
9d8a8c6f13 Resolve issues while running Automatic update of daily frequency data (from yahoo finance) for US region (#1358)
* Update YahooNormalizeUS1dExtend(#1196)

* Prevent pandas read_csv errors while running update_data_to_bin for US region

* Fix parse_index error while running update_data_to_bin for US region

* prevent pandas.read_csv error on specific symbol names

* Reordering parameters for better rendering

* removes prefix during feature_dir existence checking

* add explanation comments
2022-12-05 14:50:28 +08:00
Maxim Smolskiy
5b73b80293 Fix the Errors/Warnings when building Qlib's documentation (#1381)
* Fix the Errors/Warnings when building Qlib's documentation

* Fix

* Fix

* Empty

* Test CI

* Add doc compiling checking to CI

* Fix

* Tries to be consistent with Makefile

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2022-12-05 09:29:03 +08:00
YQ Tsui
6a47416a2d Fix logging_level: make logging level specified in qlib.init applies to all loggers (#1368)
* fix logging_level: make logging level specified in qlib.init apply to all loggers

* downgrade loglevel in expmanager __init__ to debug (it will be called in each process in multiprocessing operations such as read data)

* correct gramma error

* fix black lint

* use functor to cache loggers and set level

* correct black lint

* correct pylint

* correct pylint
2022-11-29 08:09:22 +08:00
Di
7e5bab599a Add early stopping to double ensemble model, add example (#1375)
* Add early stopping to double ensemble model, add example

* Fix lint error
2022-11-28 14:02:44 +08:00
Linlang
c4ee9ff882 Fixed log_param error (#1362)
* fix_qrun_error

* add_description
2022-11-20 14:18:35 +08:00
YQ Tsui
cc01812c62 Fix typos and grammar errors in docstrings and comments (#1366)
* fix gramma error in doc strings

* fix typos in exchange.py

* fix typos and gramma errors

* fix typo and rename function param to avoid shading python keyword

* remove redundant parathesis; pass kwargs to parent class

* fix pyblack

* further correction

* assign -> be assigned to
2022-11-20 14:15:59 +08:00
Chia-hung Tai
0c4db8b0f8 Set _artifact_uri when mlflow_run is not None. (#1367)
* Set _artifact_uri when mlflow_run is not None.

* Fix black.
2022-11-19 11:56:30 +08:00
you-n-g
994f89319d Optimize the implementation of uri & Fix async log bug (#1364)
* Optimize the implementation of uri

* remove redundant func

* Set the right order of _set_client_uri

* Update qlib/workflow/expm.py

* Simplify client & add test.Add docs; Fix async bug

* Fix comments & pylint

* Improve README
2022-11-18 13:11:31 +08:00
Maxim Smolskiy
b51e881be3 Fix the Errors with unexpected indentation when building Qlib's documentation (#1352)
* Fix ERROR: Unexpected indentation in qlib/data/dataset/handler.py

* Fix ERROR: Unexpected indentation in qlib/data/dataset/__init__.py

* Fix ERROR: Unexpected indentation in ../qlib/data/cache.py

* Fix ERROR: Unexpected indentation in qlib/model/meta/task.py

* Fix ERROR: Unexpected indentation in qlib/model/meta/dataset.py

* Fix ERROR: Unexpected indentation in qlib/workflow/online/manager.py

* Fix ERROR: Unexpected indentation in qlib/workflow/online/update.py

* Fix ERROR: Unexpected indentation in /qlib/workflow/__init__.py

* Fix ERROR: Unexpected indentation in qlib/data/base.py

* Fix ERROR: Unexpected indentation in qlib/data/dataset/loader.py

* Fix ERROR: Unexpected indentation in qlib/contrib/evaluate.py

* Fix ERROR: Unexpected indentation in qlib/workflow/record_temp.py

* Fix ERROR: Unexpected indentation in qlib/workflow/task/gen.py

* Fix ERROR: Unexpected indentation in qlib/strategy/base.py

* Fix qlib/data/dataset/handler.py

* Retest
2022-11-15 08:49:36 +08:00
qianyun210603
4001a5d157 Bug fix for Rank and WMA operators (#1228)
* bug fix: 1) 100 should be used to scale down percentileofscore return to 0-1, not length of array; 2) for (linear) weighted MA(n), weight should be n, n-1, ..., 1 instead of n-1, ..., 0

* use native pandas fucntion for rank

* remove useless import

* require pandas 1.4+

* rank for py37+pandas 1.3.5 compatibility

* lint improvement

* lint black fix

* use hasattr instead of version to check whether rolling.rank is implemented
2022-11-13 19:03:23 +08:00
He Yi
ff2154c618 fix bug in fix clip_outlier in class RobustZScoreNorm(Processor) (#1294) 2022-11-11 19:53:33 +08:00
Xu Yang
a82cc0b129 update TSDataSampler refineing the memory layout of data array to speed up NN training (#1342)
* update TSDataSampler

* reformat code with black

* use pre-commit to reformat the code

* Add documents

* More docstring

* More Safety

Co-authored-by: Young <afe.young@gmail.com>
2022-11-11 19:35:10 +08:00
you-n-g
3b471a0fe3 Fix CI (#1347) 2022-11-11 10:25:04 +08:00
Lewen Wang
e182124e75 Add docs for qlib.rl (#1322)
* Add docs for qlib.rl

* Update docs for qlib.rl

* Add homepage introduct to RL framework

* Update index Link

* Fix Icon

* typo

* Update catelog

* Update docs for qlib.rl

* Update docs for qlib.rl

* Update figure

* Update docs for qlib.rl

* Update setup.py

* FIx setup.py

* Update docs and fix some typos

* Fix the reference to RL docs

* Update framework.svg

* Update framework.svg

* Update framework.svg

* Update docs for qlibrl.

* Update docs for qlibrl.

* Update docs for Qlibrl.

* Update docs for qlibrl.

* Update docs for qlibrl.

* Update docs for qlibrl.

* Add new framework

* Update jpg

* Update framework.svg

* Update framework.svg

* Update Qlib framework and description

* Update grammar

* Update README.md

* Update README.md

* Update docs/component/rl.rst

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>

* Update docs/component/rl.rst

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>

* Update docs for qlib.rl

* Change theme for docs.

* Update docs for qlib.rl

* Update docs for qlib.rl

* Update docs for qlib.rl

* Update docs for qlib.rl.

* Update docs for qlib.rl

* Update docs for qlib.rl

* Update docs for qlib.rl

Co-authored-by: Young <afe.young@gmail.com>
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2022-11-10 21:10:44 +08:00
Huoran Li
35794846ff Refine RL todos (#1332)
* Refine several todos

* CI issues

* Remove Dropna limitation of `quote_df` in Exchange  (#1334)

* Remove Dropna limitation of `quote_df` of Exchange

* Impreove docstring

* Fix type error when expression is specified (#1335)

* Refine fill_missing_data()

* Remove several TODO comments

* Add back env for interpreters

* Change Literal import

* Resolve PR comments

* Move  to SAOEState

* Add Trainer.get_policy_state_dict()

* Mypy issue

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2022-11-10 21:10:11 +08:00
Jinge Wang
49a5bccfec Don't disable existing logger when initializing qlib. (#1339)
* Don't disable existing logger when initializing qlib.

* Add comma in the end of the config line.

* Add comment to the added config.

Co-authored-by: Jinge Wang <jingewang@microsoft.com>
2022-11-10 09:20:33 +08:00
lerit
59fbf23a71 fix position access error (#1267)
* fix position access error

position is s sub attribute of _value
error since commit(id:89972f6c6f9fa629b4f74093d4ba1e93c9f7a5e5)

* lint with blank
2022-11-08 10:51:43 +08:00
qianyun210603
94e420f755 Correct errors and typos in doc strings (#1338)
* add missing parameters to doc string in order_generate

* fix some typos in doc strings

* reformat base on code style standard

* Update qlib/backtest/__init__.py

* Update examples/run_all_model.py

* Update examples/run_all_model.py

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2022-11-07 23:37:18 +08:00
wony
67d618d4b2 Update cache.py (#1329)
make D.feature([symbol], [Feature('close')], disk_cache=1) work correctly
2022-11-03 17:08:22 +08:00
Chia-hung Tai
fb5888be9e Use mock data for element operator tests. (#1330) 2022-10-30 16:27:59 +08:00
Linlang
1861c8edaf optimize_CI (#1314) 2022-10-20 08:38:05 +08:00