1
0
mirror of https://github.com/microsoft/qlib.git synced 2026-07-14 08:16:54 +08:00

update docs link & readme.md

This commit is contained in:
bxdd
2020-09-24 13:45:26 +08:00
parent 2572284d20
commit b2749d2e08
11 changed files with 81 additions and 81 deletions

View File

@@ -11,14 +11,14 @@ With Qlib, you can easily try your ideas to create better Quant investment strat
For more details, please refer to our paper ["Qlib: An AI-oriented Quantitative Investment Platform"](https://arxiv.org/abs/2009.11189). For more details, please refer to our paper ["Qlib: An AI-oriented Quantitative Investment Platform"](https://arxiv.org/abs/2009.11189).
- [Framework of Qlib](#framework-of-qlib) - [Framework of Qlib](#Framework-of-Qlib)
- [Quick Start](#Quick-Start) - [Quick Start](#Quick-Start)
- [Installation](#Installation) - [Installation](#Installation)
- [Data Preparation](#Data-Preparation) - [Data Preparation](#Data-Preparation)
- [Auto Quant Research Workflow with](#Auto-Quant-Research-Workflow) - [Auto Quant Research Workflow with](#Auto-Quant-Research-Workflow)
- [Building Customized Quant Research Workflow by Code](#Building-Customized-Quant-Research-Workflow-by-Code) - [Building Customized Quant Research Workflow by Code](#Building-Customized-Quant-Research-Workflow-by-Code)
- [More About Qlib](#More-About-Qlib) - [More About Qlib](#More-About-Qlib)
- [Offline mode and online mode of data server](#Offline-Mode-and-Online-Mode-of-the-Data-Server) - [Offline Mode and Online Mode](#Offline-Mode-and-Online-Mode)
- [Performance of Qlib Data Server](#Performance-of-Qlib-Data-Server) - [Performance of Qlib Data Server](#Performance-of-Qlib-Data-Server)
- [Contributing](#Contributing) - [Contributing](#Contributing)
@@ -183,12 +183,12 @@ Qlib is in active and continuing development. Our plan is in the roadmap, which
# Offline Mode and Online Mode of the Data Server # Offline Mode and Online Mode
The data server of Qlib can either deployed as offline mode or online mode. The default mode is offline mode. The data server of Qlib can either deployed as `Offline` mode or `Online` mode. The default mode is offline mode.
Under offline mode, the data will be deployed locally. Under `Offline` mode, the data will be deployed locally.
Under online mode, the data will be deployed as a shared data service. The data and their cache will be shared by all the clients. The data retrieval performance is expected to be improved due to a higher rate of cache hits. It will consume less disk space, too. The documents of the online mode can be found in [Qlib-Server](https://qlib-server.readthedocs.io/). The online mode can be deployed automatically with [Azure CLI based scripts](https://qlib-server.readthedocs.io/en/latest/build.html#one-click-deployment-in-azure). The source code of online data server can be found in [qlib-server repository](https://github.com/microsoft/qlib-server). Under `Online` mode, the data will be deployed as a shared data service. The data and their cache will be shared by all the clients. The data retrieval performance is expected to be improved due to a higher rate of cache hits. It will consume less disk space, too. The documents of the online mode can be found in [Qlib-Server](https://qlib-server.readthedocs.io/). The online mode can be deployed automatically with [Azure CLI based scripts](https://qlib-server.readthedocs.io/en/latest/build.html#one-click-deployment-in-azure). The source code of online data server can be found in [Qlib-Server repository](https://github.com/microsoft/qlib-server).
## Performance of Qlib Data Server ## Performance of Qlib Data Server
The performance of data processing is important to data-driven methods like AI technologies. As an AI-oriented platform, Qlib provides a solution for data storage and data processing. To demonstrate the performance of Qlib data server, we The performance of data processing is important to data-driven methods like AI technologies. As an AI-oriented platform, Qlib provides a solution for data storage and data processing. To demonstrate the performance of Qlib data server, we

View File

@@ -49,52 +49,53 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
.. code-block:: python .. code-block:: python
>>> from qlib.contrib.estimator.handler import QLibDataHandler >> from qlib.contrib.estimator.handler import QLibDataHandler
>>> fields = ['(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close'] # MACD >> MACD_EXP = '(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close'
>>> names = ['MACD'] >> fields = [MACD_EXP] # MACD
>>> labels = ['Ref($vwap, -2)/Ref($vwap, -1) - 1'] # label >> names = ['MACD']
>>> label_names = ['LABEL'] >> labels = ['$close'] # label
>>> data_handler = QLibDataHandler(start_date='2010-01-01', end_date='2017-12-31', fields=fields, names=names, labels=labels, label_names=label_names) >> label_names = ['LABEL']
>>> TRAINER_CONFIG = { >> data_handler = QLibDataHandler(start_date='2010-01-01', end_date='2017-12-31', fields=fields, names=names, labels=labels, label_names=label_names)
... "train_start_date": "2007-01-01", >> TRAINER_CONFIG = {
... "train_end_date": "2014-12-31", .. "train_start_date": "2007-01-01",
... "validate_start_date": "2015-01-01", .. "train_end_date": "2014-12-31",
... "validate_end_date": "2016-12-31", .. "validate_start_date": "2015-01-01",
... "test_start_date": "2017-01-01", .. "validate_end_date": "2016-12-31",
... "test_end_date": "2020-08-01", .. "test_start_date": "2017-01-01",
... } .. "test_end_date": "2020-08-01",
>>> feature_train, label_train, feature_validate, label_validate, feature_test, label_test = data_handler.get_split_data(**TRAINER_CONFIG) .. }
>>> print(feature_train, label_train) >> feature_train, label_train, feature_validate, label_validate, feature_test, label_test = data_handler.get_split_data(**TRAINER_CONFIG)
MACD >> print(feature_train, label_train)
MACD
instrument datetime instrument datetime
SH600004 2012-01-04 -0.030853 SH600000 2010-01-04 -0.008625
2012-01-05 -0.030452 2010-01-05 -0.007234
2012-01-06 -0.028252 2010-01-06 -0.007693
2012-01-09 -0.024507 2010-01-07 -0.009633
2012-01-10 -0.019744 2010-01-08 -0.009891
... ... ... ...
SZ300273 2014-12-25 0.031339 SZ300251 2014-12-25 0.043072
2014-12-26 0.029695 2014-12-26 0.041345
2014-12-29 0.025577 2014-12-29 0.042733
2014-12-30 0.020493 2014-12-30 0.042066
2014-12-31 0.017089 2014-12-31 0.036299
[605882 rows x 1 columns] [322025 rows x 1 columns]
label LABEL
instrument datetime instrument datetime
SH600004 2012-01-04 0.003021 SH600000 2010-01-04 4.260015
2012-01-05 0.017434 2010-01-05 4.292182
2012-01-06 0.015490 2010-01-06 4.207747
2012-01-09 0.002324 2010-01-07 4.113258
2012-01-10 -0.002542 2010-01-08 4.159496
... ... ... ...
SZ300273 2014-12-25 -0.032454 SZ300251 2014-12-25 4.343212
2014-12-26 -0.016638 2014-12-26 4.470587
2014-12-29 0.008263 2014-12-29 4.762474
2014-12-30 -0.011985 2014-12-30 4.369748
2014-12-31 0.047797 2014-12-31 4.182222
[605882 rows x 1 columns] [322025 rows x 1 columns]
Reference Reference
=========== ===========

View File

@@ -31,9 +31,9 @@ The simple example of the default strategy is as follows.
# pred_score is the prediction score # pred_score is the prediction score
report, positions = backtest(pred_score, topk=50, n_drop=0.5, verbose=False, limit_threshold=0.0095) report, positions = backtest(pred_score, topk=50, n_drop=0.5, verbose=False, limit_threshold=0.0095)
To know more about backtesting with a specific strategy, please refer to `Strategy <strategy.html>`_. To know more about backtesting with a specific ``Strategy``, please refer to `Strategy <strategy.html>`_.
To know more about the prediction score `pred_score` output by ``Model``, please refer to `Interday Model: Model Training & Prediction <model.html>`_. To know more about the prediction score `pred_score` output by ``Interday Model``, please refer to `Interday Model: Model Training & Prediction <model.html>`_.
Prediction Score Prediction Score
----------------- -----------------
@@ -58,7 +58,7 @@ A prediction sample is shown as follows.
SH603133 2019-04-30 -0.302460 SH603133 2019-04-30 -0.302460
SZ300760 2019-04-30 -0.126383 SZ300760 2019-04-30 -0.126383
``Model`` module can make predictions, please refer to `Model <model.html>`_. ``Interday Model`` module can make predictions, please refer to `Interday Model: Model Training & Prediction <model.html>`_.
Backtest Result Backtest Result
------------------ ------------------
@@ -110,4 +110,4 @@ The backtest results are in the following form:
Reference Reference
============== ==============
To know more about ``Intraday Trading``, please refer to `Backtest API <../reference/api.html>`_. To know more about ``Intraday Trading``, please refer to `Intraday Trading <../reference/api.html#module-qlib.contrib.evaluate>`_.

View File

@@ -124,9 +124,9 @@ Feature
- `ExpressionOps` - `ExpressionOps`
`ExpressionOps` will use operator for feature construction. `ExpressionOps` will use operator for feature construction.
To know more about ``Operator``, please refer to `Operator API <../reference/api.html>`_. To know more about ``Operator``, please refer to `Operator API <../reference/api.html#module-qlib.data.ops>`_.
To know more about ``Feature``, please refer to `Feature API <../reference/api.html>`_. To know more about ``Feature``, please refer to `Feature API <../reference/api.html#module-qlib.data.base>`_.
Filter Filter
------------------- -------------------
@@ -142,25 +142,25 @@ Filter
- `cross-sectional features filter` : rule_expression = '$rank($close)<10' - `cross-sectional features filter` : rule_expression = '$rank($close)<10'
- `time-sequence features filter`: rule_expression = '$Ref($close, 3)>100' - `time-sequence features filter`: rule_expression = '$Ref($close, 3)>100'
To know more about ``Filter``, please refer to `Filter API <../reference/api.html>`_. To know more about ``Filter``, please refer to `Filter API <../reference/api.html#module-qlib.data.filter>`_.
API Reference
------------- -------------
To know more about ``Data API``, please refer to `Data API <../reference/api.html>`_. To know more about ``Data API``, please refer to `Data API <../reference/api.html#data>`_.
Data Handler Data Handler
================= =================
Users can use ``Data Handler`` in an automatic workflow by ``Estimator``, refer to `Estimator <estimator.html>`_ for more details. Users can use ``Data Handler`` in an automatic workflow by ``Estimator``, refer to `Estimator: Workflow Management <estimator.html>`_ for more details.
Also, ``Data Handler`` can be used as an independent module, by which users can easily preprocess data(standardization, remove NaN, etc.) and build datasets. It is a subclass of ``qlib.contrib.estimator.handler.BaseDataHandler``, which provides some interfaces as follows. Also, ``Data Handler`` can be used as an independent module, by which users can easily preprocess data(standardization, remove NaN, etc.) and build datasets. It is a subclass of ``qlib.contrib.estimator.handler.BaseDataHandler``, which provides some interfaces as follows.
Base Class & Interface Base Class & Interface
---------------------- ----------------------
Qlib provides a base class `qlib.contrib.estimator.BaseDataHandler <../reference/api.html#class-qlib.contrib.estimator.BaseDataHandler>`_, which provides the following interfaces: Qlib provides a base class `qlib.contrib.estimator.BaseDataHandler <../reference/api.html#qlib.contrib.estimator.handler.BaseDataHandler>`_, which provides the following interfaces:
- `setup_feature` - `setup_feature`
Implement the interface to load the data features. Implement the interface to load the data features.
@@ -204,7 +204,7 @@ Example
``Data Handler`` can be run with ``estimator`` by modifying the configuration file, and can also be used as a single module. ``Data Handler`` can be run with ``estimator`` by modifying the configuration file, and can also be used as a single module.
Know more about how to run ``Data Handler`` with ``estimator``, please refer to `Estimator <estimator.html#about-data>`_. Know more about how to run ``Data Handler`` with ``Estimator``, please refer to `Estimator: Workflow Management <estimator.html>`_
Qlib provides implemented data handler `QLibDataHandlerClose`. The following example shows how to run `QLibDataHandlerV1` as a single module. Qlib provides implemented data handler `QLibDataHandlerClose`. The following example shows how to run `QLibDataHandlerV1` as a single module.
@@ -243,14 +243,14 @@ Qlib provides implemented data handler `QLibDataHandlerClose`. The following exa
print(x_train, y_train, x_validate, y_validate, x_test, y_test) print(x_train, y_train, x_validate, y_validate, x_test, y_test)
.. note:: (x_train, y_train, x_validate, y_validate, x_test, y_test) can be used as arguments for the ``fit``, ``predict``, and ``score`` methods of the 'Model' , please refer to `Model <model.html#Interface>`_. .. note:: (x_train, y_train, x_validate, y_validate, x_test, y_test) can be used as arguments for the `fit`, `predic``, and `score` methods of the ``Interday Model`` , please refer to `Model <model.html#base-class-interface>`_.
Also, the above example has been given in ``examples.estimator.train_backtest_analyze.ipynb``. Also, the above example has been given in ``examples.estimator.train_backtest_analyze.ipynb``.
API API
--------- ---------
To know more about ``Data Handler``, please refer to `Data Handler API <../reference/api.html#handler>`_. To know more about ``Data Handler``, please refer to `Data Handler API <../reference/api.html#module-qlib.contrib.estimator.handler>`_.
Cache Cache
========== ==========
@@ -336,5 +336,3 @@ We've specially designed a file structure to manage data and cache, please refer
- .index : an assorted index file recording the line index of all calendars - .index : an assorted index file recording the line index of all calendars
- ... - ...
.. TODO: refer to paper

View File

@@ -279,7 +279,7 @@ Qlib supports custom models, but it must be a subclass of the `qlib.contrib.mode
The class `SomeModel` should be in the module `custom_model`, and ``Qlib`` could parse the `module_path` to load the class. The class `SomeModel` should be in the module `custom_model`, and ``Qlib`` could parse the `module_path` to load the class.
To know more about ``Model``, please refer to `Model <model.html>`_. To know more about ``Interday Model``, please refer to `Interday Model: Training & Prediction <model.html>`_.
Data Section Data Section
----------------- -----------------
@@ -552,7 +552,7 @@ Users can specify `backtest` through a config file, for example:
Backtest initial cash, integer type. The `account` in `strategy` section is deprecated. It only works when `account` is not set in `backtest` section. It will be overridden by `account` in the `backtest` section. The default value is 1e9. Backtest initial cash, integer type. The `account` in `strategy` section is deprecated. It only works when `account` is not set in `backtest` section. It will be overridden by `account` in the `backtest` section. The default value is 1e9.
- `deal_price` - `deal_price`
Order transaction price field, str type, the default value is vwap. Order transaction price field, str type, the default value is close.
- `min_cost` - `min_cost`
Min transaction cost, float type, the default value is 5. Min transaction cost, float type, the default value is 5.
@@ -586,7 +586,7 @@ Experiment Result
Form of Experimental Result Form of Experimental Result
---------------------------- ----------------------------
The result of the experiment is also the result of the ``Interdat Trading(Backtest)``, please refer to `Interday Trading <backtest.html>`_. The result of the experiment is also the result of the ``Intraday Trading(Backtest)``, please refer to `Intraday Trading: Model&Strategy Testing <backtest.html>`_.
Get Experiment Result Get Experiment Result

View File

@@ -6,7 +6,7 @@ Interday Model: Model Training & Prediction
Introduction Introduction
=================== ===================
``Interday Model`` is designed to make the `prediction score` about stocks. Users can use the ``Interday Model`` in an automatic workflow by ``Estimator``, please refer to `Estimator <estimator.html>`_. ``Interday Model`` is designed to make the `prediction score` about stocks. Users can use the ``Interday Model`` in an automatic workflow by ``Estimator``, please refer to `Estimator: Workflow Management <estimator.html>`_.
Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Model`` can be used as an independent module also. Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Model`` can be used as an independent module also.
@@ -48,7 +48,7 @@ The base class provides the following interfaces:
.. note:: .. note::
The number and names of the columns are determined by the data handler, please refer to `Data Handler <data.html#data-handler>`_ and `Estimator Data <estimator.html#about-data>`_. The number and names of the columns are determined by the data handler, please refer to `Data Handler <data.html#data-handler>`_ and `Estimator Data Section <estimator.html#data-section>`_.
- `y_train`, pd.DataFrame type, train label - `y_train`, pd.DataFrame type, train label
The following example explains the value of `y_train`: The following example explains the value of `y_train`:
@@ -117,7 +117,7 @@ Example
``Qlib`` provides ``LightGBM`` and ``DNN`` models as the baseline, the following steps show how to run`` LightGBM`` as an independent module. ``Qlib`` provides ``LightGBM`` and ``DNN`` models as the baseline, the following steps show how to run`` LightGBM`` as an independent module.
- Initialize ``Qlib`` with `qlib.init` first, please refer to `initialization <../start/initialization.html>`_. - Initialize ``Qlib`` with `qlib.init` first, please refer to `Initialization <../start/initialization.html>`_.
- Run the following code to get the `prediction score` `pred_score` - Run the following code to get the `prediction score` `pred_score`
.. code-block:: Python .. code-block:: Python
@@ -157,7 +157,6 @@ Example
"num_threads": 20, "num_threads": 20,
} }
# use default model # use default model
# custom Model, refer to: TODO: Model API url
model = LGBModel(**MODEL_CONFIG) model = LGBModel(**MODEL_CONFIG)
model.fit(x_train, y_train, x_validate, y_validate) model.fit(x_train, y_train, x_validate, y_validate)
_pred = model.predict(x_test) _pred = model.predict(x_test)

View File

@@ -100,7 +100,7 @@ Graphical Result
- Axis Y: - Axis Y:
- `ic` - `ic`
The `Pearson correlation coefficient` series between `label` and `prediction score`. The `Pearson correlation coefficient` series between `label` and `prediction score`.
In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. Please refer to `Data API Featrue <data.html>`_ for more details. In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. Please refer to `Data Featrue <data.html#feature>`_ for more details.
- `rank_ic` - `rank_ic`
The `Spearman's rank correlation coefficient` series between `label` and `prediction score`. The `Spearman's rank correlation coefficient` series between `label` and `prediction score`.

View File

@@ -7,7 +7,7 @@ Interday Strategy: Portfolio Management
Introduction Introduction
=================== ===================
``Interday Strategy`` is designed to adopt different trading strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Interday Model``. Users can use the ``Interday Strategy`` in an automatic workflow by ``Estimator``, please refer to `Estimator <estimator.html>`_. ``Interday Strategy`` is designed to adopt different trading strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Interday Model``. Users can use the ``Interday Strategy`` in an automatic workflow by ``Estimator``, please refer to `Estimator: Workflow Management <estimator.html>`_.
Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Strategy`` can be used as an independent module also. Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Strategy`` can be used as an independent module also.
@@ -95,11 +95,13 @@ Usage & Example
"limit_threshold": 0.095, "limit_threshold": 0.095,
"account": 100000000, "account": 100000000,
"benchmark": BENCHMARK, "benchmark": BENCHMARK,
"deal_price": "vwap", "deal_price": "close",
} "open_cost": 0.0005,
"close_cost": 0.0015,
"min_cost": 5,
}
# use default strategy # use default strategy
# custom Strategy, refer to: TODO: Strategy API url
strategy = TopkDropoutStrategy(**STRATEGY_CONFIG) strategy = TopkDropoutStrategy(**STRATEGY_CONFIG)
# pred_score is the `prediction score` output by Model # pred_score is the `prediction score` output by Model
@@ -115,4 +117,4 @@ To know more about ``Intraday Trading``, please refer to `Intraday Trading: Mode
Reference Reference
=================== ===================
To know more about ``Interday Strategy``, please refer to `Strategy API <../reference/api.html>`_. To know more about ``Interday Strategy``, please refer to `Strategy API <../reference/api.html#module-qlib.contrib.strategy.strategy>`_.

View File

@@ -44,7 +44,7 @@ Load and prepare data by running the following code:
This dataset is created by public data collected by crawler scripts in ``scripts/data_collector/``, which have been released in the same repository. Users could create the same dataset with it. This dataset is created by public data collected by crawler scripts in ``scripts/data_collector/``, which have been released in the same repository. Users could create the same dataset with it.
To kown more about `prepare data`, please refer to `Data Preparation <../component/data.html>`_. To kown more about `prepare data`, please refer to `Data Preparation <../component/data.html#data-preparation>`_.
Auto Quant Research Workflow Auto Quant Research Workflow
==================================== ====================================
@@ -60,7 +60,7 @@ Auto Quant Research Workflow
- Estimator result - Estimator result
The result of ``Estimator`` is as follows, which is also the result of ``Interday Trading``. Please refer to please refer to `Interdat Trading <../component/backtest.html>`_. for more details about the result. The result of ``Estimator`` is as follows, which is also the result of ``Intraday Trading``. Please refer to `Intraday Trading <../component/backtest.html>`_. for more details about the result.
.. code-block:: python .. code-block:: python
@@ -77,7 +77,7 @@ Auto Quant Research Workflow
max_drawdown -0.075024 max_drawdown -0.075024
To know more about `Estimator`, please refer to `Estimator <../component/estimator.html>`_. To know more about `Estimator`, please refer to `Estimator: Workflow Management <../component/estimator.html>`_.
- Graphical Reports Analysis: - Graphical Reports Analysis:
- Run ``examples/estimator/analyze_from_estimator.ipynb`` with jupyter notebook - Run ``examples/estimator/analyze_from_estimator.ipynb`` with jupyter notebook

View File

@@ -15,7 +15,7 @@ Please follow the steps below to initialize ``Qlib``.
.. code-block:: bash .. code-block:: bash
python scripts/get_data.py qlib_data_cn --target_dir ~/.qlib/qlib_data/cn_data python scripts/get_data.py qlib_data_cn --target_dir ~/.qlib/qlib_data/cn_data
Please refer to `Raw Data <../component/data.html>`_ for more information about ``get_data.py``, Please refer to `Data Preparation <../component/data.html#data-preparation>`_ for more information about `get_data.py`,
- Initialize Qlib before calling other APIs: run following code in python. - Initialize Qlib before calling other APIs: run following code in python.

View File

@@ -63,7 +63,7 @@ The Custom models need to inherit `qlib.contrib.model.base.Model <../reference/a
- Override the `predict` method - Override the `predict` method
- The parameters include the test features. - The parameters include the test features.
- Return the `prediction score`. - Return the `prediction score`.
- Please refer to `qlib.contrib.model.base.Model <../reference/api.html#module-qlib.contrib.model.base>`_ for the parameter types of the fit method. - Please refer to `Model API <../reference/api.html#module-qlib.contrib.model.base>`_ for the parameter types of the fit method.
- Code Example: In the following example, users need to use dnn to predict the label(such as `preds`) of test data `x_test` and return it. - Code Example: In the following example, users need to use dnn to predict the label(such as `preds`) of test data `x_test` and return it.
.. code-block:: Python .. code-block:: Python
@@ -143,4 +143,4 @@ Also, ``Model`` can also be tested as a single module. An example has been given
Reference Reference
===================== =====================
To know more about ``Model``, please refer to `Interday Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.contrib.model.base>`_. To know more about ``Interday Model``, please refer to `Interday Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.contrib.model.base>`_.