Huoran Li
35794846ff
Refine RL todos ( #1332 )
...
* Refine several todos
* CI issues
* Remove Dropna limitation of `quote_df` in Exchange (#1334 )
* Remove Dropna limitation of `quote_df` of Exchange
* Impreove docstring
* Fix type error when expression is specified (#1335 )
* Refine fill_missing_data()
* Remove several TODO comments
* Add back env for interpreters
* Change Literal import
* Resolve PR comments
* Move to SAOEState
* Add Trainer.get_policy_state_dict()
* Mypy issue
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2022-11-10 21:10:11 +08:00
Chia-hung Tai
fb5888be9e
Use mock data for element operator tests. ( #1330 )
2022-10-30 16:27:59 +08:00
Linlang
1861c8edaf
optimize_CI ( #1314 )
2022-10-20 08:38:05 +08:00
Chia-hung Tai
84e9df2603
Add REG_US and REG_TW into test case: test_utils.py. ( #1310 )
...
* Add REG_US and REG_TW into test case: test_utils.py.
* Fix black.
* Trigger checks.
* Add REG_US and REG_TW into test case: test_utils.py.
* Fix black.
* Trigger checks.
2022-10-14 11:49:30 +08:00
Huoran Li
216a8ec2de
RL backtest with simulator ( #1299 )
...
* RL backtest with simulator
* Minor modification in init_qlib
* Cherry pick PR 1302
* Resolve PR comments
* Fix missing data processing
* Minor bugfix
* Add TODOs and docs
* Add a comment
2022-10-12 16:44:28 +08:00
Huoran Li
bee05f56ef
Migrate backtest logic from NT ( #1263 )
...
* Backtest migration
* Minor bug fix in test
* Reorganize file to avoid loop import
* Fix test SAOE bug
* Remove unnecessary names
* Resolve PR comments; remove private classes;
* Fix CI error
* Resolve PR comments
* Refactor data interfaces
* Remove convert_instance_config and change config
* Pylint issue
* Pylint issue
* Fix tempfile warning
* Resolve PR comments
* Add more comments
2022-09-19 14:54:26 +08:00
Huoran Li
1d65d28b28
Qlib simulator refinement (redo of PR 1244) ( #1262 )
...
* Use dict-like configuration
* Rename from_neutrader to integration
* SAOE strategy
* Optimize file structure
* Optimize code
* Format code
* create_state_maintainer_recursive
* Remove explicit time_per_step
* CI test passed
* Resolve PR comments
* Pass all CI
* Minor test issue
* Refine SAOE adapter logic
* Minor bugfix
* Cherry pick updates
* Resolve PR comments
* CI issues
* Refine adapter & saoe_data logic
* Resolve PR comments
* Resolve PR comments
* Rename ONE_SEC to EPS_T; complete backtest loop
* CI issue
* Resolve Yuge's PR comments
2022-08-24 14:09:45 +08:00
Huoran Li
2752bdc92c
Migrate NeuTrader to Qlib RL ( #1169 )
...
* Refine previous version RL codes
* Polish utils/__init__.py
* Draft
* Use | instead of Union
* Simulator & action interpreter
* Test passed
* Migrate to SAOEState & new qlib interpreter
* Black format
* . Revert file_storage change
* Refactor file structure & renaming functions
* Enrich test cases
* Add QlibIntradayBacktestData
* Test interpreter
* Black format
* .
.
.
* Rename receive_execute_result()
* Use indicator to simplify state update
* Format code
* Modify data path
* Adjust file structure
* Minor change
* Add copyright message
* Format code
* Rename util functions
* Add CI
* Pylint issue
* Remove useless code to pass pylint
* Pass mypy
* Mypy issue
* mypy issue
* mypy issue
* Revert "mypy issue"
This reverts commit 8eb1b0174eafe.young@gmail.com >
2022-08-01 09:56:07 +08:00
you-n-g
a6700d81ff
Update test_qlib_from_source_slow.yml's timeout setting. ( #1178 )
...
* Update test_qlib_from_source_slow.yml
* Update test_qlib_from_source.yml
* Update test_pit.py
* Update test_pit.py
* Update test_pit.py
* Update test_pit.py
2022-07-06 20:44:10 +08:00
Chao Wang
3db22452fb
Adding ChangeInstrument op ( #1005 )
...
* add ChangeInstrument to ops
Adding Change instrument OP. This op allows one to use features of a different instrument.
* Update __init__.py
update parse_field to accommodate ChangeInstrument
* Propose test
* Add test case and fix bug
* Update ops.py
* Update ops.py
* simplify the operator further
* implement abstract method
* fix arg bug
* clean test
Co-authored-by: Young <afe.young@gmail.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2022-07-04 08:45:26 +08:00
Yuge Zhang
25ecb1135f
Qlib RL framework (stage 2) - trainer ( #1125 )
...
* checkpoint
(cherry picked from commit 1a8e0bd4671ee6d624a7d09bb198a273282cd050)
* Not a workable version
(cherry picked from commit 3498e185684cd5590d3ab97e0ab69eab8c1e0e3a)
* vessel
* ckpt
* .
* vessel
* .
* .
* checkpoint callback
* .
* cleanup
* logger
* .
* test
* .
* add test
* .
* .
* .
* .
* New reward
* Add train API
* fix mypy
* fix lint
* More comment
* 3.7 compat
* fix test
* fix test
* .
* Resolve comments
* fix typehint
2022-06-28 19:53:05 +08:00
Linlang
2ca0d88d2d
change_pitdata_source ( #1171 )
...
* change_pitdata_source
* retain_normalize
* add_comment
2022-06-28 16:29:59 +08:00
Linlang
50d74b5560
split_CI ( #1141 )
2022-06-28 10:17:29 +08:00
Huoran Li
89972f6c6f
Refine backtest codes ( #1120 )
...
* Refine backtest code
* Keep working
* Minor
* Resolve PR comments
* Fix import error
* Fix import error
2022-06-10 12:14:48 +08:00
you-n-g
cc94c32db6
init_instance_by_config enhancement ( #1103 )
...
* fix SepDataFrame when we del it to empty
* init_instance_by_config enhancement
* Update test_sepdf.py
2022-05-21 20:16:22 +08:00
Yuge Zhang
9a40fd3cdc
Qlib RL framework (stage 1) - single-asset order execution ( #1076 )
...
* rl init
* aux info
* Reward config
* update
* simple
* update saoe init
* update simulator and seed
* minor
* minor
* update sim
* checkpoint
* obs
* Update interpreter
* init qlib simulator
* checkpoint
* Refine codebase
* checkpoint
* checkpoint
* Add one test
* More tests
* Simulator checkpoint
* checkpoint
* First-step tested
* Checkpoint
* Update data_queue API
* Checkpoint
* Update test
* Move files
* Checkpoint
* Single-quote -> double-quote
* Fix finite env tests
* Tested with mypy
* pep-574
* No call for env done
* Update finite env docs
* Fix csv writer
* Refine tester
* Update logger
* Add another logger test
* Checkpoint
* Add network sanity test
* steps per episode is not correct
* Cleanup code, ready for PR
* Reformat with black
* Fix pylint for py37
* Fix lint
* Fix lint
* Fix flake
* update mypy command
* mypy
* Update exclude pattern
* Use pyproject.toml
* test
* .
* .
* Refactor pipeline
* .
* defaults run bash
* .
* Revert and skip follow_imports
* Fix toml issue
* fix mypy
* .
* .
* .
* Fix install
* Minor fix
* Fix test
* Fix test
* Remove requirements
* Revert
* fix tests
* Fix lint
* .
* .
* .
* .
* .
* update install from source command
* .
* Fix data download
* .
* .
* .
* .
* .
* .
* Fix py37
* Ignore tests on non-linux
* resolve comments
* fix tests
* resolve comments
* some typo
* style updates
* More comments
* fix dummy
* add warning
* Align precision in some system
* Added some impl notes
Co-authored-by: Young <afe.young@gmail.com >
2022-05-21 18:19:24 +08:00
Linlang
2cf842bcfe
add_test_pit ( #1089 )
...
* add_test_pit
* add_test_pit_to_tests
* add_baostock_to_setup
* add_pip_to_CI
Co-authored-by: Linlang Lv (iSoftStone) <v-linlanglv@microsoft.com >
2022-05-06 16:47:20 +08:00
you-n-g
c5cf8fb9cc
fix est_sepdf.py with black
2022-04-29 17:21:20 +08:00
you-n-g
3c9c76b384
fix SepDataFrame when we del it to empty ( #1082 )
2022-04-29 14:29:17 +08:00
you-n-g
9d0a8f61d1
Make sepdf more like DataFrame ( #1080 )
2022-04-28 19:13:45 +08:00
Chia-hung Tai
07f0d4f599
[930] Fix typo HasingStockStorage to HashingStockStorage. ( #947 )
2022-03-04 12:40:04 +08:00
you-n-g
60d45ad770
Enhance pytorch nn ( #917 )
...
* enhance pytorch_nn
* fix dim bug
* Black format
* Fix pylint error
2022-02-15 19:22:48 +08:00
luocy16
2bb8a4ce0e
Supporting Arctic Backend Provider & Orderbook, Tick Data Example ( #744 )
...
* change weight_decay & batchsize
* del weight_decay
* big weight_decay
* mid weight_decay
* small layer
* 2 layer
* full layer
* no weight decay
* divide into two data source
* change parse field
* delete some debug
* add Toperator
* new format of arctic
* fix cache bug to arctic read
* fix connection problem
* add some operator
* final version for arcitc
* clear HZ cache
* remove not used function
* add topswrappers
* successfully import data and run first test
* A simpler version to support arctic
* Successfully run all high-freq expressions
* Black format and fix add docs
* Add docs for download and test data
* update scripts and docs
* Add docs
* fix bug
* Refine docs
* fix test bug
* fix CI error
* clean code
Co-authored-by: bxdd <bxddream@gmail.com >
Co-authored-by: wangwenxi.handsome <wangwenxi.handsome@gmail.com >
Co-authored-by: Young <afe.young@gmail.com >
2022-01-18 09:13:11 +08:00
Chia-hung Tai
edd8badeaf
[840] - Test case for operators. ( #841 )
...
* [840] - Test case for operators.
* Move import to the head of file and add test_setting.
2022-01-11 18:44:15 +08:00
Chia-hung Tai
184ce34a34
[807] Move the REG_CONSTANT/EPS to constant.py. ( #811 )
...
* [807] Move the REG_CONSTANT to constant.py.
* import REG_US.
* Move EPS to constant.py.
2022-01-09 21:39:46 +08:00
Pengrong Zhu
c276de4040
Fix backtest ( #719 )
...
* modify FileStorage to support multiple freqs
* modify backtest's sample documentation
* change the logging level of read data exception from error to debug
* fix the backtest exception when volume is 0 or np.nan
* fix test_storage.py
* add backtest_daily
* modify backtest_daily's docstring
* add __repr__/__str__ to Position
* fix the bug of nested_decision_execution example
Co-authored-by: Young <afe.young@gmail.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
2021-12-07 19:04:23 +08:00
you-n-g
45ebb1d0e0
support adding from date when updating pred ( #703 )
...
* support adding from date when updating pred
* fix updating data error
2021-11-22 19:52:52 +08:00
you-n-g
cae4c9c924
An example to get index from TSDataSampler ( #679 )
2021-11-10 14:35:27 +08:00
Young
3fa48d7017
simplify record tmp
2021-11-05 12:57:14 +00:00
Young
64130d9407
Fix the aggregation function of IndexData
2021-10-22 15:20:45 +08:00
Young
052aad7982
simplify signal parameter
2021-10-15 14:48:31 +00:00
Young
ac08468330
Make static prediction easier
2021-10-15 11:21:03 +00:00
Young
28fe4d4bb4
update file strategy test
2021-10-03 14:58:37 +08:00
Young
873129aa9b
update fix CI tests bugs
2021-10-03 14:58:37 +08:00
Young
4099050935
Merge remote-tracking branch 'origin/main' into nested_decision_exe
2021-09-30 18:41:15 +00:00
wangwenxi-handsome
3760a18a8d
Merge nested main ( #597 )
...
* MVP for Indian Stocks in qlib using yahooquery
* cleaned with black
* cleaned with black
* add YahooNormalizeIN and YahooNormalizeIN1d
* cleaned the code
* added 1min for IN and also updated readme
* update comments
* fix comments
* recorder support upload both raw file and directory
* fix comments
* Update README.md
* Fix docs of QlibRecorder
* sort index after loader (#538 )
make sure the fetch method is based on a index-sorted pd.DataFrame
* refactor online serving rolling api
* refactor TRA
* format by black
* fix horizon
* fix TRA when use single head
* clean up
* improve pretrain
* update README
* fix tra when logdir is None
* fix tra when logdir is None
* Update strategy.py
* Update README.md
* Update README.md
* Conda Suggestion
* code standard docs
* Update ensemble.py (#560 )
* Fix CI Bug (#575 )
Co-authored-by: yuxwang <anduinnn@foxmail.com >
* Update gen.py (#576 )
* Fix multi-process loop calls (#574 )
* check lexsort in the 'lazy_sort_index' function (#566 )
* check lexsort
* check lexsort
* lexsort comment
* lexsort comment
* Delete .DS_Store
* Update README.md
* bug fix & use oracle transport pretrain
* mend
* Add `backend_freq_config` parameter, support multi-freq uri
* Add sample_config to QlibDataLoader, support multi-freq
* add multi-freq example
* get_cls_kwargs renamed get_callable_kwargs
* support multi-freq uri
* Add inst_processors to D.features
* Fix typo
* Fix the index type of the multi-freq example
* Fix duplicate mlflow directories in tests
* Add DataPathManager to QlibConfig && modify inst_processors to supports list only
* Modify the default value in the multi_freq example
* Modify client-server mode and dataset-cache to disable inst_processor
* Add wheel package to github CI
* fix comment
* Update FAQ.rst
* Update README.md
Fix wrong link
* Update the docs of TaskManager (#586 )
* Update manage.py
* update yaml
* update run_all_model
* Modify the Feature to be case sensitive (#589 )
* update README
* remove verbose
* fix spell bug
* fix typos (#592 )
* Update Release Note
* fix portfolio bug
* Add calendar support for resample
* add freq kwargs
* test.yml: Remove redundant code (#595 )
* Supporting shared processor (#596 )
* Supporting shared processor
* fix readonly reverse bug
* remove pytests dependency
* with fit bug
* fix parameter error
* fix comments
* Fix undefined names in Python code (#599 )
* Update pytorch_tabnet.py
$ `flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics`
```
./qlib/qlib/contrib/model/pytorch_tabnet.py:567:38: F821 undefined name 'inp'
self.independ.append(GLU(inp, out_dim, vbs=vbs))
^
./qlib/examples/model_rolling/task_manager_rolling.py:75:18: F821 undefined name 'task_train'
run_task(task_train, self.task_pool, experiment_name=self.experiment_name)
^
2 F821 undefined name 'task_train'
2
```
* Fix undefined names in Python code
* from qlib.model.trainer import task_train
* update seed
* fix some docstring
* add comments
* Fix SimpleDatasetCache
* Update setup.py
updated classifiers
* Update setup.py
change to matplotlib==3.3
* Update python-publish.yml
added python 3.9
* updategrade version number
* Update model list
* fix the type of filter_pipe
* fix comment
* fix record_temp
* update cvxpy version
* Update code_standard.rst (#587 )
* Update code_standard.rst
* Update docs/developer/code_standard.rst
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
* Add file lock for MLflowExpManager (#619 )
* fix torch version
* Share version number (#620 )
* Update initialization.rst (#622 )
* Update initialization.rst
* Update docs/start/initialization.rst
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
* Update docs/start/initialization.rst
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
* fix bugs for running previous exmaple
* fix deal amount bug
* update change doc (#623 )
* Add files via upload
* Update README.md
* Update README.md
* Update README.md
* Delete change doc.gif
* Add files via upload
* Update README.md
* Delete change doc.gif
* Add files via upload
* Delete change doc.gif
* Add files via upload
* Update README.md
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
* update doc
* simplify run all model
* fix run all model bug
* Fix Models (#483 )
* fix gat dataset
* fix tft model
* Update tft.py
* Fix tft.py
Co-authored-by: Pengrong Zhu <zhu.pengrong@foxmail.com >
* type and skip empty exp
* fix model yaml config
* fix tft import bug
* skip empty result
* fix model and yaml bug
* fix wrong generate parameter
* Modify multi-freq example (#626 )
* modify the example of multi-freq
* add Copyright
* add a comment to average_ops.py
* modify the example of multi-freq
* add comment to multi_freq_handler.py
* add the Ref expression description to multi_freq_handler.py
* add expression description to multi_freq_handler.py
* update images
* fix workflow and update framework
Co-authored-by: Gaurav <2796gaurav@gmail.com >
Co-authored-by: 2796gaurav <17353992+2796gaurav@users.noreply.github.com >
Co-authored-by: bxdd <bxd98@126.com >
Co-authored-by: Young <afe.young@gmail.com >
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com >
Co-authored-by: Dong Zhou <Zhou.Dong@microsoft.com >
Co-authored-by: ZhangTP1996 <ztp18@mails.tsinghua.edu.cn >
Co-authored-by: demon143 <59681577+demon143@users.noreply.github.com >
Co-authored-by: Wangwuyi123 <51237097+Wangwuyi123@users.noreply.github.com >
Co-authored-by: yuxwang <anduinnn@foxmail.com >
Co-authored-by: Pengrong Zhu <zhu.pengrong@foxmail.com >
Co-authored-by: Mark Zhao <50850474+markzhao98@users.noreply.github.com >
Co-authored-by: cslwqxx <cslwqxx@users.noreply.github.com >
Co-authored-by: Dong Zhou <evanzd@users.noreply.github.com >
Co-authored-by: SaintMalik <37118134+saintmalik@users.noreply.github.com >
Co-authored-by: Christian Clauss <cclauss@me.com >
Co-authored-by: Anurag Kumar <mailanu98@gmail.com >
Co-authored-by: demon143 <785696300@qq.com >
2021-10-01 02:15:30 +08:00
you-n-g
b9809a4c33
make the prediction update more friendly ( #609 )
...
* make the prediction update more friendly
* Update test_storage.py
* LabelUpdater
* Update test_storage.py
* Update test_storage.py
* Update test_storage.py
* Update test_storage.py
* Update setup.py
* Update workflow_config_lightgbm_Alpha158.yaml
* Update workflow_config_lightgbm_Alpha158.yaml
* Update workflow_config_lightgbm_Alpha158.yaml
* Update workflow_config_lightgbm_Alpha158.yaml
* Update workflow_config_lightgbm_Alpha158.yaml
* Update setup.py
* Update setup.py
* test CI only
* test CI only
* Update workflow_config_lightgbm_Alpha158.yaml
* Update setup.py
* fix "Segmentation fault" in macos
* Update test.yml
github action no longer supported ubuntu-16.04
* Update api.rst
update doc with new_lable
* Update api.rst
Co-authored-by: Wangwuyi123 <51237097+Wangwuyi123@users.noreply.github.com >
Co-authored-by: Pengrong Zhu <zhu.pengrong@foxmail.com >
2021-09-30 20:54:44 +08:00
you-n-g
163e3c6266
replace multi processing with joblib ( #477 )
...
* replace multi processing with joblib
* update class Parallel and data.py
* update class Parallel and data.py
* update class Parallel and data.py
* update class Parallel and data.py
* update class Parallel and data.py
* update class Parallel and data.py
* update class Parallel and data.py
* update class Parallel and data.py
* Fix Parallel support for maxtasksperchild
Co-authored-by: wangw <1666490690@qq.com >
Co-authored-by: zhupr <zhu.pengrong@foxmail.com >
2021-09-14 01:16:03 +08:00
you-n-g
51709c20d8
Supporting shared processor ( #596 )
...
* Supporting shared processor
* fix readonly reverse bug
* remove pytests dependency
* with fit bug
* fix parameter error
2021-09-13 17:11:08 +08:00
zhupr
707399a245
Fix duplicate mlflow directories in tests
2021-09-07 20:41:10 +08:00
Young
88d2f9263e
fix sum index data bug
2021-09-02 01:57:44 +00:00
Young
5003e49197
fix metric calculation error
2021-09-02 09:56:38 +08:00
Young
5f0ee6ce68
fix bugs
2021-09-02 09:56:38 +08:00
Young
9a74471ab6
Pass basic tests
2021-09-02 09:56:38 +08:00
Young
d39c8de800
draft design
2021-09-02 09:56:38 +08:00
wangwenxi.handsome
f67b99a30e
update exchange
2021-09-02 09:56:38 +08:00
Young
309dfa36cc
Add a example to collecting all the decisions
2021-08-15 15:22:48 +00:00
wangwenxi-handsome
735153a50d
Cash Update ( #559 )
...
* fix negative cash
* update order test
* fix bug
* update file_order_test
2021-08-12 23:44:22 +08:00
wangwenxi.handsome
7c858803f0
add position test
2021-08-08 14:32:33 +00:00
Young
0c946cffd6
add supporting setting trade unit in exchange
2021-07-07 10:47:54 +00:00
