liuyu/qlib
1
0
Fork 0
mirror of https://github.com/microsoft/qlib.git synced 2026-06-06 05:51:17 +08:00
Code Activity

update docs link & readme.md

Browse Source
This commit is contained in:
bxdd
2020-09-24 13:45:26 +08:00
parent 2572284d20
commit b2749d2e08
11 changed files with 81 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@@ -11,14 +11,14 @@ With Qlib, you can easily try your ideas to create better Quant investment strat
For more details, please refer to our paper ["Qlib: An AI-oriented Quantitative Investment Platform"](https://arxiv.org/abs/2009.11189).
- [Framework of Qlib](#framework-of-qlib)
- [Framework of Qlib](#Framework-of-Qlib)
- [Quick Start](#Quick-Start)
- [Installation](#Installation)
- [Data Preparation](#Data-Preparation)
- [Auto Quant Research Workflow with](#Auto-Quant-Research-Workflow)
- [Building Customized Quant Research Workflow by Code](#Building-Customized-Quant-Research-Workflow-by-Code)
- [More About Qlib](#More-About-Qlib)
- [Offline mode and online mode of data server](#Offline-Mode-and-Online-Mode-of-the-Data-Server)
- [Offline Mode and Online Mode](#Offline-Mode-and-Online-Mode)
- [Performance of Qlib Data Server](#Performance-of-Qlib-Data-Server)
- [Contributing](#Contributing)
@@ -183,12 +183,12 @@ Qlib is in active and continuing development. Our plan is in the roadmap, which
# Offline Mode and Online Mode of the Data Server
The data server of Qlib can either deployed as offline mode or online mode. The default mode is offline mode.
# Offline Mode and Online Mode
The data server of Qlib can either deployed as `Offline` mode or `Online` mode. The default mode is offline mode.
Under offline mode, the data will be deployed locally.
Under `Offline` mode, the data will be deployed locally.
Under online mode, the data will be deployed as a shared data service. The data and their cache will be shared by all the clients. The data retrieval performance is expected to be improved due to a higher rate of cache hits. It will consume less disk space, too. The documents of the online mode can be found in [Qlib-Server](https://qlib-server.readthedocs.io/). The online mode can be deployed automatically with [Azure CLI based scripts](https://qlib-server.readthedocs.io/en/latest/build.html#one-click-deployment-in-azure). The source code of online data server can be found in [qlib-server repository](https://github.com/microsoft/qlib-server).
Under `Online` mode, the data will be deployed as a shared data service. The data and their cache will be shared by all the clients. The data retrieval performance is expected to be improved due to a higher rate of cache hits. It will consume less disk space, too. The documents of the online mode can be found in [Qlib-Server](https://qlib-server.readthedocs.io/). The online mode can be deployed automatically with [Azure CLI based scripts](https://qlib-server.readthedocs.io/en/latest/build.html#one-click-deployment-in-azure). The source code of online data server can be found in [Qlib-Server repository](https://github.com/microsoft/qlib-server).
## Performance of Qlib Data Server
The performance of data processing is important to data-driven methods like AI technologies. As an AI-oriented platform, Qlib provides a solution for data storage and data processing. To demonstrate the performance of Qlib data server, we

81
docs/advanced/alpha.rst
View File

@@ -49,52 +49,53 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
.. code-block:: python
>>> from qlib.contrib.estimator.handler import QLibDataHandler
>>> fields = ['(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close'] # MACD
>>> names = ['MACD']
>>> labels = ['Ref($vwap, -2)/Ref($vwap, -1) - 1'] # label
>>> label_names = ['LABEL']
>>> data_handler = QLibDataHandler(start_date='2010-01-01', end_date='2017-12-31', fields=fields, names=names, labels=labels, label_names=label_names)
>>> TRAINER_CONFIG = {
... "train_start_date": "2007-01-01",
... "train_end_date": "2014-12-31",
... "validate_start_date": "2015-01-01",
... "validate_end_date": "2016-12-31",
... "test_start_date": "2017-01-01",
... "test_end_date": "2020-08-01",
... }
>>> feature_train, label_train, feature_validate, label_validate, feature_test, label_test = data_handler.get_split_data(**TRAINER_CONFIG)
>>> print(feature_train, label_train)
MACD
>> from qlib.contrib.estimator.handler import QLibDataHandler
>> MACD_EXP = '(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close'
>> fields = [MACD_EXP] # MACD
>> names = ['MACD']
>> labels = ['$close'] # label
>> label_names = ['LABEL']
>> data_handler = QLibDataHandler(start_date='2010-01-01', end_date='2017-12-31', fields=fields, names=names, labels=labels, label_names=label_names)
>> TRAINER_CONFIG = {
.. "train_start_date": "2007-01-01",
.. "train_end_date": "2014-12-31",
.. "validate_start_date": "2015-01-01",
.. "validate_end_date": "2016-12-31",
.. "test_start_date": "2017-01-01",
.. "test_end_date": "2020-08-01",
.. }
>> feature_train, label_train, feature_validate, label_validate, feature_test, label_test = data_handler.get_split_data(**TRAINER_CONFIG)
>> print(feature_train, label_train)
MACD
instrument datetime
SH600004 2012-01-04 -0.030853
2012-01-05 -0.030452
2012-01-06 -0.028252
2012-01-09 -0.024507
2012-01-10 -0.019744
SH600000 2010-01-04 -0.008625
2010-01-05 -0.007234
2010-01-06 -0.007693
2010-01-07 -0.009633
2010-01-08 -0.009891
... ...
SZ300273 2014-12-25 0.031339
2014-12-26 0.029695
2014-12-29 0.025577
2014-12-30 0.020493
2014-12-31 0.017089
SZ300251 2014-12-25 0.043072
2014-12-26 0.041345
2014-12-29 0.042733
2014-12-30 0.042066
2014-12-31 0.036299
[605882 rows x 1 columns]
label
[322025 rows x 1 columns]
LABEL
instrument datetime
SH600004 2012-01-04 0.003021
2012-01-05 0.017434
2012-01-06 0.015490
2012-01-09 0.002324
2012-01-10 -0.002542
SH600000 2010-01-04 4.260015
2010-01-05 4.292182
2010-01-06 4.207747
2010-01-07 4.113258
2010-01-08 4.159496
... ...
SZ300273 2014-12-25 -0.032454
2014-12-26 -0.016638
2014-12-29 0.008263
2014-12-30 -0.011985
2014-12-31 0.047797
SZ300251 2014-12-25 4.343212
2014-12-26 4.470587
2014-12-29 4.762474
2014-12-30 4.369748
2014-12-31 4.182222
[605882 rows x 1 columns]
[322025 rows x 1 columns]
Reference
===========

8
docs/component/backtest.rst
View File

@@ -31,9 +31,9 @@ The simple example of the default strategy is as follows.
# pred_score is the prediction score
report, positions = backtest(pred_score, topk=50, n_drop=0.5, verbose=False, limit_threshold=0.0095)
To know more about backtesting with a specific strategy, please refer to `Strategy <strategy.html>`_.
To know more about backtesting with a specific ``Strategy``, please refer to `Strategy <strategy.html>`_.
To know more about the prediction score `pred_score` output by ``Model``, please refer to `Interday Model: Model Training & Prediction <model.html>`_.
To know more about the prediction score `pred_score` output by ``Interday Model``, please refer to `Interday Model: Model Training & Prediction <model.html>`_.
Prediction Score
-----------------
@@ -58,7 +58,7 @@ A prediction sample is shown as follows.
SH603133 2019-04-30 -0.302460
SZ300760 2019-04-30 -0.126383
``Model`` module can make predictions, please refer to `Model <model.html>`_.
``Interday Model`` module can make predictions, please refer to `Interday Model: Model Training & Prediction <model.html>`_.
Backtest Result
------------------
@@ -110,4 +110,4 @@ The backtest results are in the following form:
Reference
==============
To know more about ``Intraday Trading``, please refer to `Backtest API <../reference/api.html>`_.
To know more about ``Intraday Trading``, please refer to `Intraday Trading <../reference/api.html#module-qlib.contrib.evaluate>`_.

22
docs/component/data.rst
View File

@@ -124,9 +124,9 @@ Feature
- `ExpressionOps`
`ExpressionOps` will use operator for feature construction.
To know more about ``Operator``, please refer to `Operator API <../reference/api.html>`_.
To know more about ``Operator``, please refer to `Operator API <../reference/api.html#module-qlib.data.ops>`_.
To know more about ``Feature``, please refer to `Feature API <../reference/api.html>`_.
To know more about ``Feature``, please refer to `Feature API <../reference/api.html#module-qlib.data.base>`_.
Filter
-------------------
@@ -142,25 +142,25 @@ Filter
- `cross-sectional features filter` : rule_expression = '$rank($close)<10'
- `time-sequence features filter`: rule_expression = '$Ref($close, 3)>100'
To know more about ``Filter``, please refer to `Filter API <../reference/api.html>`_.
To know more about ``Filter``, please refer to `Filter API <../reference/api.html#module-qlib.data.filter>`_.
API
Reference
-------------
To know more about ``Data API``, please refer to `Data API <../reference/api.html>`_.
To know more about ``Data API``, please refer to `Data API <../reference/api.html#data>`_.
Data Handler
=================
Users can use ``Data Handler`` in an automatic workflow by ``Estimator``, refer to `Estimator <estimator.html>`_ for more details.
Users can use ``Data Handler`` in an automatic workflow by ``Estimator``, refer to `Estimator: Workflow Management <estimator.html>`_ for more details.
Also, ``Data Handler`` can be used as an independent module, by which users can easily preprocess data(standardization, remove NaN, etc.) and build datasets. It is a subclass of ``qlib.contrib.estimator.handler.BaseDataHandler``, which provides some interfaces as follows.
Base Class & Interface
----------------------
Qlib provides a base class `qlib.contrib.estimator.BaseDataHandler <../reference/api.html#class-qlib.contrib.estimator.BaseDataHandler>`_, which provides the following interfaces:
Qlib provides a base class `qlib.contrib.estimator.BaseDataHandler <../reference/api.html#qlib.contrib.estimator.handler.BaseDataHandler>`_, which provides the following interfaces:
- `setup_feature`
Implement the interface to load the data features.
@@ -204,7 +204,7 @@ Example
``Data Handler`` can be run with ``estimator`` by modifying the configuration file, and can also be used as a single module.
Know more about how to run ``Data Handler`` with ``estimator``, please refer to `Estimator <estimator.html#about-data>`_.
Know more about how to run ``Data Handler`` with ``Estimator``, please refer to `Estimator: Workflow Management <estimator.html>`_
Qlib provides implemented data handler `QLibDataHandlerClose`. The following example shows how to run `QLibDataHandlerV1` as a single module.
@@ -243,14 +243,14 @@ Qlib provides implemented data handler `QLibDataHandlerClose`. The following exa
print(x_train, y_train, x_validate, y_validate, x_test, y_test)
.. note:: (x_train, y_train, x_validate, y_validate, x_test, y_test) can be used as arguments for the ``fit``, ``predict``, and ``score`` methods of the 'Model' , please refer to `Model <model.html#Interface>`_.
.. note:: (x_train, y_train, x_validate, y_validate, x_test, y_test) can be used as arguments for the `fit`, `predic``, and `score` methods of the ``Interday Model`` , please refer to `Model <model.html#base-class-interface>`_.
Also, the above example has been given in ``examples.estimator.train_backtest_analyze.ipynb``.
API
---------
To know more about ``Data Handler``, please refer to `Data Handler API <../reference/api.html#handler>`_.
To know more about ``Data Handler``, please refer to `Data Handler API <../reference/api.html#module-qlib.contrib.estimator.handler>`_.
Cache
==========
@@ -336,5 +336,3 @@ We've specially designed a file structure to manage data and cache, please refer
- .index : an assorted index file recording the line index of all calendars
- ...
.. TODO: refer to paper

6
docs/component/estimator.rst
View File

@@ -279,7 +279,7 @@ Qlib supports custom models, but it must be a subclass of the `qlib.contrib.mode
The class `SomeModel` should be in the module `custom_model`, and ``Qlib`` could parse the `module_path` to load the class.
To know more about ``Model``, please refer to `Model <model.html>`_.
To know more about ``Interday Model``, please refer to `Interday Model: Training & Prediction <model.html>`_.
Data Section
-----------------
@@ -552,7 +552,7 @@ Users can specify `backtest` through a config file, for example:
Backtest initial cash, integer type. The `account` in `strategy` section is deprecated. It only works when `account` is not set in `backtest` section. It will be overridden by `account` in the `backtest` section. The default value is 1e9.
- `deal_price`
Order transaction price field, str type, the default value is vwap.
Order transaction price field, str type, the default value is close.
- `min_cost`
Min transaction cost, float type, the default value is 5.
@@ -586,7 +586,7 @@ Experiment Result
Form of Experimental Result
----------------------------
The result of the experiment is also the result of the ``Interdat Trading(Backtest)``, please refer to `Interday Trading <backtest.html>`_.
The result of the experiment is also the result of the ``Intraday Trading(Backtest)``, please refer to `Intraday Trading: Model&Strategy Testing <backtest.html>`_.
Get Experiment Result

7
docs/component/model.rst
View File

@@ -6,7 +6,7 @@ Interday Model: Model Training & Prediction
Introduction
===================
``Interday Model`` is designed to make the `prediction score` about stocks. Users can use the ``Interday Model`` in an automatic workflow by ``Estimator``, please refer to `Estimator <estimator.html>`_.
``Interday Model`` is designed to make the `prediction score` about stocks. Users can use the ``Interday Model`` in an automatic workflow by ``Estimator``, please refer to `Estimator: Workflow Management <estimator.html>`_.
Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Model`` can be used as an independent module also.
@@ -48,7 +48,7 @@ The base class provides the following interfaces:
.. note::
The number and names of the columns are determined by the data handler, please refer to `Data Handler <data.html#data-handler>`_ and `Estimator Data <estimator.html#about-data>`_.
The number and names of the columns are determined by the data handler, please refer to `Data Handler <data.html#data-handler>`_ and `Estimator Data Section <estimator.html#data-section>`_.
- `y_train`, pd.DataFrame type, train label
The following example explains the value of `y_train`:
@@ -117,7 +117,7 @@ Example
``Qlib`` provides ``LightGBM`` and ``DNN`` models as the baseline, the following steps show how to run`` LightGBM`` as an independent module.
- Initialize ``Qlib`` with `qlib.init` first, please refer to `initialization <../start/initialization.html>`_.
- Initialize ``Qlib`` with `qlib.init` first, please refer to `Initialization <../start/initialization.html>`_.
- Run the following code to get the `prediction score` `pred_score`
.. code-block:: Python
@@ -157,7 +157,6 @@ Example
"num_threads": 20,
}
# use default model
# custom Model, refer to: TODO: Model API url
model = LGBModel(**MODEL_CONFIG)
model.fit(x_train, y_train, x_validate, y_validate)
_pred = model.predict(x_test)

2
docs/component/report.rst
View File

@@ -100,7 +100,7 @@ Graphical Result
- Axis Y:
- `ic`
The `Pearson correlation coefficient` series between `label` and `prediction score`.
In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. Please refer to `Data API Featrue <data.html>`_ for more details.
In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. Please refer to `Data Featrue <data.html#feature>`_ for more details.
- `rank_ic`
The `Spearman's rank correlation coefficient` series between `label` and `prediction score`.

12
docs/component/strategy.rst
View File

@@ -7,7 +7,7 @@ Interday Strategy: Portfolio Management
Introduction
===================
``Interday Strategy`` is designed to adopt different trading strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Interday Model``. Users can use the ``Interday Strategy`` in an automatic workflow by ``Estimator``, please refer to `Estimator <estimator.html>`_.
``Interday Strategy`` is designed to adopt different trading strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Interday Model``. Users can use the ``Interday Strategy`` in an automatic workflow by ``Estimator``, please refer to `Estimator: Workflow Management <estimator.html>`_.
Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Strategy`` can be used as an independent module also.
@@ -95,11 +95,13 @@ Usage & Example
"limit_threshold": 0.095,
"account": 100000000,
"benchmark": BENCHMARK,
"deal_price": "vwap",
"deal_price": "close",
"open_cost": 0.0005,
"close_cost": 0.0015,
"min_cost": 5,
}
# use default strategy
# custom Strategy, refer to: TODO: Strategy API url
strategy = TopkDropoutStrategy(**STRATEGY_CONFIG)
# pred_score is the `prediction score` output by Model
@@ -115,4 +117,4 @@ To know more about ``Intraday Trading``, please refer to `Intraday Trading: Mode
Reference
===================
To know more about ``Interday Strategy``, please refer to `Strategy API <../reference/api.html>`_.
To know more about ``Interday Strategy``, please refer to `Strategy API <../reference/api.html#module-qlib.contrib.strategy.strategy>`_.

6
docs/introduction/quick.rst
View File

@@ -44,7 +44,7 @@ Load and prepare data by running the following code:
This dataset is created by public data collected by crawler scripts in ``scripts/data_collector/``, which have been released in the same repository. Users could create the same dataset with it.
To kown more about `prepare data`, please refer to `Data Preparation <../component/data.html>`_.
To kown more about `prepare data`, please refer to `Data Preparation <../component/data.html#data-preparation>`_.
Auto Quant Research Workflow
====================================
@@ -60,7 +60,7 @@ Auto Quant Research Workflow
- Estimator result
The result of ``Estimator`` is as follows, which is also the result of ``Interday Trading``. Please refer to please refer to `Interdat Trading <../component/backtest.html>`_. for more details about the result.
The result of ``Estimator`` is as follows, which is also the result of ``Intraday Trading``. Please refer to `Intraday Trading <../component/backtest.html>`_. for more details about the result.
.. code-block:: python
@@ -77,7 +77,7 @@ Auto Quant Research Workflow
max_drawdown -0.075024
To know more about `Estimator`, please refer to `Estimator <../component/estimator.html>`_.
To know more about `Estimator`, please refer to `Estimator: Workflow Management <../component/estimator.html>`_.
- Graphical Reports Analysis:
- Run ``examples/estimator/analyze_from_estimator.ipynb`` with jupyter notebook

2
docs/start/initialization.rst
View File

@@ -15,7 +15,7 @@ Please follow the steps below to initialize ``Qlib``.
.. code-block:: bash
python scripts/get_data.py qlib_data_cn --target_dir ~/.qlib/qlib_data/cn_data
Please refer to `Raw Data <../component/data.html>`_ for more information about ``get_data.py``,
Please refer to `Data Preparation <../component/data.html#data-preparation>`_ for more information about `get_data.py`,
- Initialize Qlib before calling other APIs: run following code in python.

4
docs/start/integration.rst
View File

@@ -63,7 +63,7 @@ The Custom models need to inherit `qlib.contrib.model.base.Model <../reference/a
- Override the `predict` method
- The parameters include the test features.
- Return the `prediction score`.
- Please refer to `qlib.contrib.model.base.Model <../reference/api.html#module-qlib.contrib.model.base>`_ for the parameter types of the fit method.
- Please refer to `Model API <../reference/api.html#module-qlib.contrib.model.base>`_ for the parameter types of the fit method.
- Code Example: In the following example, users need to use dnn to predict the label(such as `preds`) of test data `x_test` and return it.
.. code-block:: Python
@@ -143,4 +143,4 @@ Also, ``Model`` can also be tested as a single module. An example has been given
Reference
=====================
To know more about ``Model``, please refer to `Interday Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.contrib.model.base>`_.
To know more about ``Interday Model``, please refer to `Interday Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.contrib.model.base>`_.