mirror of
https://github.com/microsoft/qlib.git
synced 2026-07-12 23:36:54 +08:00
Improve the style of documentation (#1132)
This commit improves the documentation (rst files) only in the following three ways: * Aligned section headers with their underline/overline punctuation characters * Deleted all trailling whitespaces in rst files * Deleted a few trailling newlines at the end of the rst files Co-authored-by: Bingyao Liu <Bingyao.Liu@sofund.com>
This commit is contained in:
YaOzI
56
CHANGES.rst
56
CHANGES.rst
@@ -1,63 +1,63 @@
|
|||||||
Changelog
|
Changelog
|
||||||
====================
|
=========
|
||||||
Here you can see the full list of changes between each QLib release.
|
Here you can see the full list of changes between each QLib release.
|
||||||
|
|
||||||
Version 0.1.0
|
Version 0.1.0
|
||||||
--------------------
|
-------------
|
||||||
This is the initial release of QLib library.
|
This is the initial release of QLib library.
|
||||||
|
|
||||||
Version 0.1.1
|
Version 0.1.1
|
||||||
--------------------
|
-------------
|
||||||
Performance optimize. Add more features and operators.
|
Performance optimize. Add more features and operators.
|
||||||
|
|
||||||
Version 0.1.2
|
Version 0.1.2
|
||||||
--------------------
|
-------------
|
||||||
- Support operator syntax. Now ``High() - Low()`` is equivalent to ``Sub(High(), Low())``.
|
- Support operator syntax. Now ``High() - Low()`` is equivalent to ``Sub(High(), Low())``.
|
||||||
- Add more technical indicators.
|
- Add more technical indicators.
|
||||||
|
|
||||||
Version 0.1.3
|
Version 0.1.3
|
||||||
--------------------
|
-------------
|
||||||
Bug fix and add instruments filtering mechanism.
|
Bug fix and add instruments filtering mechanism.
|
||||||
|
|
||||||
Version 0.2.0
|
Version 0.2.0
|
||||||
--------------------
|
-------------
|
||||||
- Redesign ``LocalProvider`` database format for performance improvement.
|
- Redesign ``LocalProvider`` database format for performance improvement.
|
||||||
- Support load features as string fields.
|
- Support load features as string fields.
|
||||||
- Add scripts for database construction.
|
- Add scripts for database construction.
|
||||||
- More operators and technical indicators.
|
- More operators and technical indicators.
|
||||||
|
|
||||||
Version 0.2.1
|
Version 0.2.1
|
||||||
--------------------
|
-------------
|
||||||
- Support registering user-defined ``Provider``.
|
- Support registering user-defined ``Provider``.
|
||||||
- Support use operators in string format, e.g. ``['Ref($close, 1)']`` is valid field format.
|
- Support use operators in string format, e.g. ``['Ref($close, 1)']`` is valid field format.
|
||||||
- Support dynamic fields in ``$some_field`` format. And existing fields like ``Close()`` may be deprecated in the future.
|
- Support dynamic fields in ``$some_field`` format. And existing fields like ``Close()`` may be deprecated in the future.
|
||||||
|
|
||||||
Version 0.2.2
|
Version 0.2.2
|
||||||
--------------------
|
-------------
|
||||||
- Add ``disk_cache`` for reusing features (enabled by default).
|
- Add ``disk_cache`` for reusing features (enabled by default).
|
||||||
- Add ``qlib.contrib`` for experimental model construction and evaluation.
|
- Add ``qlib.contrib`` for experimental model construction and evaluation.
|
||||||
|
|
||||||
|
|
||||||
Version 0.2.3
|
Version 0.2.3
|
||||||
--------------------
|
-------------
|
||||||
- Add ``backtest`` module
|
- Add ``backtest`` module
|
||||||
- Decoupling the Strategy, Account, Position, Exchange from the backtest module
|
- Decoupling the Strategy, Account, Position, Exchange from the backtest module
|
||||||
|
|
||||||
Version 0.2.4
|
Version 0.2.4
|
||||||
--------------------
|
-------------
|
||||||
- Add ``profit attribution`` module
|
- Add ``profit attribution`` module
|
||||||
- Add ``rick_control`` and ``cost_control`` strategies
|
- Add ``rick_control`` and ``cost_control`` strategies
|
||||||
|
|
||||||
Version 0.3.0
|
Version 0.3.0
|
||||||
--------------------
|
-------------
|
||||||
- Add ``estimator`` module
|
- Add ``estimator`` module
|
||||||
|
|
||||||
Version 0.3.1
|
Version 0.3.1
|
||||||
--------------------
|
-------------
|
||||||
- Add ``filter`` module
|
- Add ``filter`` module
|
||||||
|
|
||||||
Version 0.3.2
|
Version 0.3.2
|
||||||
--------------------
|
-------------
|
||||||
- Add real price trading, if the ``factor`` field in the data set is incomplete, use ``adj_price`` trading
|
- Add real price trading, if the ``factor`` field in the data set is incomplete, use ``adj_price`` trading
|
||||||
- Refactor ``handler`` ``launcher`` ``trainer`` code
|
- Refactor ``handler`` ``launcher`` ``trainer`` code
|
||||||
- Support ``backtest`` configuration parameters in the configuration file
|
- Support ``backtest`` configuration parameters in the configuration file
|
||||||
@@ -65,16 +65,16 @@ Version 0.3.2
|
|||||||
- Fix bug of ``filter`` module
|
- Fix bug of ``filter`` module
|
||||||
|
|
||||||
Version 0.3.3
|
Version 0.3.3
|
||||||
-------------------
|
-------------
|
||||||
- Fix bug of ``filter`` module
|
- Fix bug of ``filter`` module
|
||||||
|
|
||||||
Version 0.3.4
|
Version 0.3.4
|
||||||
--------------------
|
-------------
|
||||||
- Support for ``finetune model``
|
- Support for ``finetune model``
|
||||||
- Refactor ``fetcher`` code
|
- Refactor ``fetcher`` code
|
||||||
|
|
||||||
Version 0.3.5
|
Version 0.3.5
|
||||||
--------------------
|
-------------
|
||||||
- Support multi-label training, you can provide multiple label in ``handler``. (But LightGBM doesn't support due to the algorithm itself)
|
- Support multi-label training, you can provide multiple label in ``handler``. (But LightGBM doesn't support due to the algorithm itself)
|
||||||
- Refactor ``handler`` code, dataset.py is no longer used, and you can deploy your own labels and features in ``feature_label_config``
|
- Refactor ``handler`` code, dataset.py is no longer used, and you can deploy your own labels and features in ``feature_label_config``
|
||||||
- Handler only offer DataFrame. Also, ``trainer`` and model.py only receive DataFrame
|
- Handler only offer DataFrame. Also, ``trainer`` and model.py only receive DataFrame
|
||||||
@@ -82,7 +82,7 @@ Version 0.3.5
|
|||||||
- Move some date config from ``handler`` to ``trainer``
|
- Move some date config from ``handler`` to ``trainer``
|
||||||
|
|
||||||
Version 0.4.0
|
Version 0.4.0
|
||||||
--------------------
|
-------------
|
||||||
- Add `data` package that holds all data-related codes
|
- Add `data` package that holds all data-related codes
|
||||||
- Reform the data provider structure
|
- Reform the data provider structure
|
||||||
- Create a server for data centralized management `qlib-server<https://amc-msra.visualstudio.com/trading-algo/_git/qlib-server>`_
|
- Create a server for data centralized management `qlib-server<https://amc-msra.visualstudio.com/trading-algo/_git/qlib-server>`_
|
||||||
@@ -100,7 +100,7 @@ Version 0.4.0
|
|||||||
|
|
||||||
|
|
||||||
Version 0.4.1
|
Version 0.4.1
|
||||||
--------------------
|
-------------
|
||||||
- Add support Windows
|
- Add support Windows
|
||||||
- Fix ``instruments`` type bug
|
- Fix ``instruments`` type bug
|
||||||
- Fix ``features`` is empty bug(It will cause failure in updating)
|
- Fix ``features`` is empty bug(It will cause failure in updating)
|
||||||
@@ -112,19 +112,19 @@ Version 0.4.1
|
|||||||
|
|
||||||
|
|
||||||
Version 0.4.2
|
Version 0.4.2
|
||||||
--------------------
|
-------------
|
||||||
- Refactor DataHandler
|
- Refactor DataHandler
|
||||||
- Add ``Alpha360`` DataHandler
|
- Add ``Alpha360`` DataHandler
|
||||||
|
|
||||||
|
|
||||||
Version 0.4.3
|
Version 0.4.3
|
||||||
--------------------
|
-------------
|
||||||
- Implementing Online Inference and Trading Framework
|
- Implementing Online Inference and Trading Framework
|
||||||
- Refactoring The interfaces of backtest and strategy module.
|
- Refactoring The interfaces of backtest and strategy module.
|
||||||
|
|
||||||
|
|
||||||
Version 0.4.4
|
Version 0.4.4
|
||||||
--------------------
|
-------------
|
||||||
- Optimize cache generation performance
|
- Optimize cache generation performance
|
||||||
- Add report module
|
- Add report module
|
||||||
- Fix bug when using ``ServerDatasetCache`` offline.
|
- Fix bug when using ``ServerDatasetCache`` offline.
|
||||||
@@ -138,7 +138,7 @@ Version 0.4.4
|
|||||||
|
|
||||||
|
|
||||||
Version 0.4.5
|
Version 0.4.5
|
||||||
--------------------
|
-------------
|
||||||
- Add multi-kernel implementation for both client and server.
|
- Add multi-kernel implementation for both client and server.
|
||||||
- Support a new way to load data from client which skips dataset cache.
|
- Support a new way to load data from client which skips dataset cache.
|
||||||
- Change the default dataset method from single kernel implementation to multi kernel implementation.
|
- Change the default dataset method from single kernel implementation to multi kernel implementation.
|
||||||
@@ -146,14 +146,14 @@ Version 0.4.5
|
|||||||
- Support a new method to write config file by using dict.
|
- Support a new method to write config file by using dict.
|
||||||
|
|
||||||
Version 0.4.6
|
Version 0.4.6
|
||||||
--------------------
|
-------------
|
||||||
- Some bugs are fixed
|
- Some bugs are fixed
|
||||||
- The default config in `Version 0.4.5` is not friendly to daily frequency data.
|
- The default config in `Version 0.4.5` is not friendly to daily frequency data.
|
||||||
- Backtest error in TopkWeightStrategy when `WithInteract=True`.
|
- Backtest error in TopkWeightStrategy when `WithInteract=True`.
|
||||||
|
|
||||||
|
|
||||||
Version 0.5.0
|
Version 0.5.0
|
||||||
--------------------
|
-------------
|
||||||
- First opensource version
|
- First opensource version
|
||||||
- Refine the docs, code
|
- Refine the docs, code
|
||||||
- Add baselines
|
- Add baselines
|
||||||
@@ -161,7 +161,7 @@ Version 0.5.0
|
|||||||
|
|
||||||
|
|
||||||
Version 0.8.0
|
Version 0.8.0
|
||||||
--------------------
|
-------------
|
||||||
- The backtest is greatly refactored.
|
- The backtest is greatly refactored.
|
||||||
- Nested decision execution framework is supported
|
- Nested decision execution framework is supported
|
||||||
- There are lots of changes for daily trading, it is hard to list all of them. But a few important changes could be noticed
|
- There are lots of changes for daily trading, it is hard to list all of them. But a few important changes could be noticed
|
||||||
@@ -175,5 +175,5 @@ Version 0.8.0
|
|||||||
|
|
||||||
|
|
||||||
Other Versions
|
Other Versions
|
||||||
----------------------------------
|
--------------
|
||||||
Please refer to `Github release Notes <https://github.com/microsoft/qlib/releases>`_
|
Please refer to `Github release Notes <https://github.com/microsoft/qlib/releases>`_
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ Qlib FAQ
|
|||||||
############
|
############
|
||||||
|
|
||||||
Qlib Frequently Asked Questions
|
Qlib Frequently Asked Questions
|
||||||
================================
|
===============================
|
||||||
.. contents::
|
.. contents::
|
||||||
:depth: 1
|
:depth: 1
|
||||||
:local:
|
:local:
|
||||||
@@ -13,7 +13,7 @@ Qlib Frequently Asked Questions
|
|||||||
|
|
||||||
|
|
||||||
1. RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase...
|
1. RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase...
|
||||||
------------------------------------------------------------------------------------------------------------------------------------
|
-----------------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
@@ -52,7 +52,7 @@ This is caused by the limitation of multiprocessing under windows OS. Please ref
|
|||||||
|
|
||||||
|
|
||||||
2. qlib.data.cache.QlibCacheException: It sees the key(...) of the redis lock has existed in your redis db now.
|
2. qlib.data.cache.QlibCacheException: It sees the key(...) of the redis lock has existed in your redis db now.
|
||||||
-----------------------------------------------------------------------------------------------------------------
|
---------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
It sees the key of the redis lock has existed in your redis db now. You can use the following command to clear your redis keys and rerun your commands
|
It sees the key of the redis lock has existed in your redis db now. You can use the following command to clear your redis keys and rerun your commands
|
||||||
|
|
||||||
@@ -72,7 +72,7 @@ If the issue is not resolved, use ``keys *`` to find if multiple keys exist. If
|
|||||||
Also, feel free to post a new issue in our GitHub repository. We always check each issue carefully and try our best to solve them.
|
Also, feel free to post a new issue in our GitHub repository. We always check each issue carefully and try our best to solve them.
|
||||||
|
|
||||||
3. ModuleNotFoundError: No module named 'qlib.data._libs.rolling'
|
3. ModuleNotFoundError: No module named 'qlib.data._libs.rolling'
|
||||||
------------------------------------------------------------------------------------------------------------------------------------
|
-----------------------------------------------------------------
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
@@ -101,7 +101,7 @@ Also, feel free to post a new issue in our GitHub repository. We always check ea
|
|||||||
|
|
||||||
|
|
||||||
4. BadNamespaceError: / is not a connected namespace
|
4. BadNamespaceError: / is not a connected namespace
|
||||||
------------------------------------------------------------------------------------------------------------------------------------
|
----------------------------------------------------
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
@@ -125,7 +125,7 @@ Also, feel free to post a new issue in our GitHub repository. We always check ea
|
|||||||
|
|
||||||
|
|
||||||
5. TypeError: send() got an unexpected keyword argument 'binary'
|
5. TypeError: send() got an unexpected keyword argument 'binary'
|
||||||
------------------------------------------------------------------------------------------------------------------------------------
|
----------------------------------------------------------------
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
|
|||||||
@@ -1,14 +1,14 @@
|
|||||||
.. _pit:
|
.. _pit:
|
||||||
|
|
||||||
===========================
|
============================
|
||||||
(P)oint-(I)n-(T)ime Database
|
(P)oint-(I)n-(T)ime Database
|
||||||
===========================
|
============================
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
------------
|
------------
|
||||||
Point-in-time data is a very important consideration when performing any sort of historical market analysis.
|
Point-in-time data is a very important consideration when performing any sort of historical market analysis.
|
||||||
|
|
||||||
For example, let’s say we are backtesting a trading strategy and we are using the past five years of historical data as our input.
|
For example, let’s say we are backtesting a trading strategy and we are using the past five years of historical data as our input.
|
||||||
Our model is assumed to trade once a day, at the market close, and we’ll say we are calculating the trading signal for 1 January 2020 in our backtest. At that point, we should only have data for 1 January 2020, 31 December 2019, 30 December 2019 etc.
|
Our model is assumed to trade once a day, at the market close, and we’ll say we are calculating the trading signal for 1 January 2020 in our backtest. At that point, we should only have data for 1 January 2020, 31 December 2019, 30 December 2019 etc.
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
.. _alpha:
|
.. _alpha:
|
||||||
|
|
||||||
===========================
|
=========================
|
||||||
Building Formulaic Alphas
|
Building Formulaic Alphas
|
||||||
===========================
|
=========================
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
|
|
||||||
In quantitative trading practice, designing novel factors that can explain and predict future asset returns are of vital importance to the profitability of a strategy. Such factors are usually called alpha factors, or alphas in short.
|
In quantitative trading practice, designing novel factors that can explain and predict future asset returns are of vital importance to the profitability of a strategy. Such factors are usually called alpha factors, or alphas in short.
|
||||||
|
|
||||||
@@ -15,28 +15,28 @@ A formulaic alpha, as the name suggests, is a kind of alpha that can be presente
|
|||||||
|
|
||||||
|
|
||||||
Building Formulaic Alphas in ``Qlib``
|
Building Formulaic Alphas in ``Qlib``
|
||||||
======================================
|
=====================================
|
||||||
|
|
||||||
In ``Qlib``, users can easily build formulaic alphas.
|
In ``Qlib``, users can easily build formulaic alphas.
|
||||||
|
|
||||||
Example
|
Example
|
||||||
-----------------
|
-------
|
||||||
|
|
||||||
`MACD`, short for moving average convergence/divergence, is a formulaic alpha used in technical analysis of stock prices. It is designed to reveal changes in the strength, direction, momentum, and duration of a trend in a stock's price.
|
`MACD`, short for moving average convergence/divergence, is a formulaic alpha used in technical analysis of stock prices. It is designed to reveal changes in the strength, direction, momentum, and duration of a trend in a stock's price.
|
||||||
|
|
||||||
`MACD` can be presented as the following formula:
|
`MACD` can be presented as the following formula:
|
||||||
|
|
||||||
.. math::
|
.. math::
|
||||||
|
|
||||||
MACD = 2\times (DIF-DEA)
|
MACD = 2\times (DIF-DEA)
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
`DIF` means Differential value, which is 12-period EMA minus 26-period EMA.
|
`DIF` means Differential value, which is 12-period EMA minus 26-period EMA.
|
||||||
|
|
||||||
.. math::
|
.. math::
|
||||||
|
|
||||||
DIF = \frac{EMA(CLOSE, 12) - EMA(CLOSE, 26)}{CLOSE}
|
DIF = \frac{EMA(CLOSE, 12) - EMA(CLOSE, 26)}{CLOSE}
|
||||||
|
|
||||||
`DEA`means a 9-period EMA of the DIF.
|
`DEA`means a 9-period EMA of the DIF.
|
||||||
|
|
||||||
@@ -65,7 +65,7 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
|||||||
>> print(df)
|
>> print(df)
|
||||||
feature label
|
feature label
|
||||||
MACD LABEL
|
MACD LABEL
|
||||||
datetime instrument
|
datetime instrument
|
||||||
2010-01-04 SH600000 -0.011547 -0.019672
|
2010-01-04 SH600000 -0.011547 -0.019672
|
||||||
SH600004 0.002745 -0.014721
|
SH600004 0.002745 -0.014721
|
||||||
SH600006 0.010133 0.002911
|
SH600006 0.010133 0.002911
|
||||||
@@ -79,7 +79,7 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
|||||||
SZ300315 -0.030557 0.012455
|
SZ300315 -0.030557 0.012455
|
||||||
|
|
||||||
Reference
|
Reference
|
||||||
===========
|
=========
|
||||||
|
|
||||||
To learn more about ``Data Loader``, please refer to `Data Loader <../component/data.html#data-loader>`_
|
To learn more about ``Data Loader``, please refer to `Data Loader <../component/data.html#data-loader>`_
|
||||||
|
|
||||||
|
|||||||
@@ -1,26 +1,26 @@
|
|||||||
.. _serial:
|
.. _serial:
|
||||||
|
|
||||||
=================================
|
=============
|
||||||
Serialization
|
Serialization
|
||||||
=================================
|
=============
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
``Qlib`` supports dumping the state of ``DataHandler``, ``DataSet``, ``Processor`` and ``Model``, etc. into a disk and reloading them.
|
``Qlib`` supports dumping the state of ``DataHandler``, ``DataSet``, ``Processor`` and ``Model``, etc. into a disk and reloading them.
|
||||||
|
|
||||||
Serializable Class
|
Serializable Class
|
||||||
========================
|
==================
|
||||||
|
|
||||||
``Qlib`` provides a base class ``qlib.utils.serial.Serializable``, whose state can be dumped into or loaded from disk in `pickle` format.
|
``Qlib`` provides a base class ``qlib.utils.serial.Serializable``, whose state can be dumped into or loaded from disk in `pickle` format.
|
||||||
When users dump the state of a ``Serializable`` instance, the attributes of the instance whose name **does not** start with `_` will be saved on the disk.
|
When users dump the state of a ``Serializable`` instance, the attributes of the instance whose name **does not** start with `_` will be saved on the disk.
|
||||||
However, users can use ``config`` method or override ``default_dump_all`` attribute to prevent this feature.
|
However, users can use ``config`` method or override ``default_dump_all`` attribute to prevent this feature.
|
||||||
|
|
||||||
Users can also override ``pickle_backend`` attribute to choose a pickle backend. The supported value is "pickle" (default and common) and "dill" (dump more things such as function, more information in `here <https://pypi.org/project/dill/>`_).
|
Users can also override ``pickle_backend`` attribute to choose a pickle backend. The supported value is "pickle" (default and common) and "dill" (dump more things such as function, more information in `here <https://pypi.org/project/dill/>`_).
|
||||||
|
|
||||||
Example
|
Example
|
||||||
==========================
|
=======
|
||||||
``Qlib``'s serializable class includes ``DataHandler``, ``DataSet``, ``Processor`` and ``Model``, etc., which are subclass of ``qlib.utils.serial.Serializable``.
|
``Qlib``'s serializable class includes ``DataHandler``, ``DataSet``, ``Processor`` and ``Model``, etc., which are subclass of ``qlib.utils.serial.Serializable``.
|
||||||
Specifically, ``qlib.data.dataset.DatasetH`` is one of them. Users can serialize ``DatasetH`` as follows.
|
Specifically, ``qlib.data.dataset.DatasetH`` is one of them. Users can serialize ``DatasetH`` as follows.
|
||||||
|
|
||||||
.. code-block:: Python
|
.. code-block:: Python
|
||||||
@@ -33,7 +33,7 @@ Specifically, ``qlib.data.dataset.DatasetH`` is one of them. Users can serialize
|
|||||||
dataset = pickle.load(file_dataset)
|
dataset = pickle.load(file_dataset)
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
Only state of ``DatasetH`` should be saved on the disk, such as some `mean` and `variance` used for data normalization, etc.
|
Only state of ``DatasetH`` should be saved on the disk, such as some `mean` and `variance` used for data normalization, etc.
|
||||||
|
|
||||||
After reloading the ``DatasetH``, users need to reinitialize it. It means that users can reset some states of ``DatasetH`` or ``QlibDataHandler`` such as `instruments`, `start_time`, `end_time` and `segments`, etc., and generate new data according to the states (data is not state and should not be saved on the disk).
|
After reloading the ``DatasetH``, users need to reinitialize it. It means that users can reset some states of ``DatasetH`` or ``QlibDataHandler`` such as `instruments`, `start_time`, `end_time` and `segments`, etc., and generate new data according to the states (data is not state and should not be saved on the disk).
|
||||||
|
|
||||||
@@ -41,5 +41,5 @@ A more detailed example is in this `link <https://github.com/microsoft/qlib/tree
|
|||||||
|
|
||||||
|
|
||||||
API
|
API
|
||||||
===================
|
===
|
||||||
Please refer to `Serializable API <../reference/api.html#module-qlib.utils.serial.Serializable>`_.
|
Please refer to `Serializable API <../reference/api.html#module-qlib.utils.serial.Serializable>`_.
|
||||||
|
|||||||
@@ -1,15 +1,15 @@
|
|||||||
.. _server:
|
.. _server:
|
||||||
|
|
||||||
=================================
|
=============================
|
||||||
``Online`` & ``Offline`` mode
|
``Online`` & ``Offline`` mode
|
||||||
=================================
|
=============================
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
=============
|
============
|
||||||
|
|
||||||
``Qlib`` supports ``Online`` mode and ``Offline`` mode. Only the ``Offline`` mode is introduced in this document.
|
``Qlib`` supports ``Online`` mode and ``Offline`` mode. Only the ``Offline`` mode is introduced in this document.
|
||||||
|
|
||||||
The ``Online`` mode is designed to solve the following problems:
|
The ``Online`` mode is designed to solve the following problems:
|
||||||
|
|
||||||
@@ -18,12 +18,12 @@ The ``Online`` mode is designed to solve the following problems:
|
|||||||
- Make the data can be accessed in a remote way.
|
- Make the data can be accessed in a remote way.
|
||||||
|
|
||||||
Qlib-Server
|
Qlib-Server
|
||||||
===============
|
===========
|
||||||
|
|
||||||
``Qlib-Server`` is the assorted server system for ``Qlib``, which utilizes ``Qlib`` for basic calculations and provides extensive server system and cache mechanism. With QLibServer, the data provided for ``Qlib`` can be managed in a centralized manner. With ``Qlib-Server``, users can use ``Qlib`` in ``Online`` mode.
|
``Qlib-Server`` is the assorted server system for ``Qlib``, which utilizes ``Qlib`` for basic calculations and provides extensive server system and cache mechanism. With QLibServer, the data provided for ``Qlib`` can be managed in a centralized manner. With ``Qlib-Server``, users can use ``Qlib`` in ``Online`` mode.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Reference
|
Reference
|
||||||
=================
|
=========
|
||||||
If users are interested in ``Qlib-Server`` and ``Online`` mode, please refer to `Qlib-Server Project <https://github.com/microsoft/qlib-server>`_ and `Qlib-Server Document <https://qlib-server.readthedocs.io/en/latest/>`_.
|
If users are interested in ``Qlib-Server`` and ``Online`` mode, please refer to `Qlib-Server Project <https://github.com/microsoft/qlib-server>`_ and `Qlib-Server Document <https://qlib-server.readthedocs.io/en/latest/>`_.
|
||||||
|
|||||||
@@ -1,13 +1,13 @@
|
|||||||
.. _task_management:
|
.. _task_management:
|
||||||
|
|
||||||
=================================
|
===============
|
||||||
Task Management
|
Task Management
|
||||||
=================================
|
===============
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
=============
|
============
|
||||||
|
|
||||||
The `Workflow <../component/introduction.html>`_ part introduces how to run research workflow in a loosely-coupled way. But it can only execute one ``task`` when you use ``qrun``.
|
The `Workflow <../component/introduction.html>`_ part introduces how to run research workflow in a loosely-coupled way. But it can only execute one ``task`` when you use ``qrun``.
|
||||||
To automatically generate and execute different tasks, ``Task Management`` provides a whole process including `Task Generating`_, `Task Storing`_, `Task Training`_ and `Task Collecting`_.
|
To automatically generate and execute different tasks, ``Task Management`` provides a whole process including `Task Generating`_, `Task Storing`_, `Task Training`_ and `Task Collecting`_.
|
||||||
@@ -36,7 +36,7 @@ Here is the base class of ``TaskGen``:
|
|||||||
This class allows users to verify the effect of data from different periods on the model in one experiment. More information is `here <../reference/api.html#TaskGen>`_.
|
This class allows users to verify the effect of data from different periods on the model in one experiment. More information is `here <../reference/api.html#TaskGen>`_.
|
||||||
|
|
||||||
Task Storing
|
Task Storing
|
||||||
===============
|
============
|
||||||
To achieve higher efficiency and the possibility of cluster operation, ``Task Manager`` will store all tasks in `MongoDB <https://www.mongodb.com/>`_.
|
To achieve higher efficiency and the possibility of cluster operation, ``Task Manager`` will store all tasks in `MongoDB <https://www.mongodb.com/>`_.
|
||||||
``TaskManager`` can fetch undone tasks automatically and manage the lifecycle of a set of tasks with error handling.
|
``TaskManager`` can fetch undone tasks automatically and manage the lifecycle of a set of tasks with error handling.
|
||||||
Users **MUST** finish the configuration of `MongoDB <https://www.mongodb.com/>`_ when using this module.
|
Users **MUST** finish the configuration of `MongoDB <https://www.mongodb.com/>`_ when using this module.
|
||||||
@@ -57,7 +57,7 @@ Users need to provide the MongoDB URL and database name for using ``TaskManager`
|
|||||||
More information of ``Task Manager`` can be found in `here <../reference/api.html#TaskManager>`_.
|
More information of ``Task Manager`` can be found in `here <../reference/api.html#TaskManager>`_.
|
||||||
|
|
||||||
Task Training
|
Task Training
|
||||||
===============
|
=============
|
||||||
After generating and storing those ``task``, it's time to run the ``task`` which is in the *WAITING* status.
|
After generating and storing those ``task``, it's time to run the ``task`` which is in the *WAITING* status.
|
||||||
``Qlib`` provides a method called ``run_task`` to run those ``task`` in task pool, however, users can also customize how tasks are executed.
|
``Qlib`` provides a method called ``run_task`` to run those ``task`` in task pool, however, users can also customize how tasks are executed.
|
||||||
An easy way to get the ``task_func`` is using ``qlib.model.trainer.task_train`` directly.
|
An easy way to get the ``task_func`` is using ``qlib.model.trainer.task_train`` directly.
|
||||||
|
|||||||
@@ -1,2 +1 @@
|
|||||||
.. include:: ../../CHANGES.rst
|
.. include:: ../../CHANGES.rst
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +1,13 @@
|
|||||||
.. _data:
|
.. _data:
|
||||||
|
|
||||||
================================
|
==================================
|
||||||
Data Layer: Data Framework & Usage
|
Data Layer: Data Framework & Usage
|
||||||
================================
|
==================================
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
============================
|
============
|
||||||
|
|
||||||
``Data Layer`` provides user-friendly APIs to manage and retrieve data. It provides high-performance data infrastructure.
|
``Data Layer`` provides user-friendly APIs to manage and retrieve data. It provides high-performance data infrastructure.
|
||||||
|
|
||||||
It is designed for quantitative investment. For example, users could build formulaic alphas with ``Data Layer`` easily. Please refer to `Building Formulaic Alphas <../advanced/alpha.html>`_ for more details.
|
It is designed for quantitative investment. For example, users could build formulaic alphas with ``Data Layer`` easily. Please refer to `Building Formulaic Alphas <../advanced/alpha.html>`_ for more details.
|
||||||
|
|
||||||
@@ -23,16 +23,16 @@ The introduction of ``Data Layer`` includes the following parts.
|
|||||||
|
|
||||||
Here is a typical example of Qlib data workflow
|
Here is a typical example of Qlib data workflow
|
||||||
|
|
||||||
- Users download data and converting data into Qlib format(with filename suffix `.bin`). In this step, typically only some basic data are stored on disk(such as OHLCV).
|
- Users download data and converting data into Qlib format(with filename suffix `.bin`). In this step, typically only some basic data are stored on disk(such as OHLCV).
|
||||||
- Creating some basic features based on Qlib's expression Engine(e.g. "Ref($close, 60) / $close", the return of last 60 trading days). Supported operators in the expression engine can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/ops.py>`_. This step is typically implemented in Qlib's `Data Loader <https://qlib.readthedocs.io/en/latest/component/data.html#data-loader>`_ which is a component of `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ .
|
- Creating some basic features based on Qlib's expression Engine(e.g. "Ref($close, 60) / $close", the return of last 60 trading days). Supported operators in the expression engine can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/ops.py>`_. This step is typically implemented in Qlib's `Data Loader <https://qlib.readthedocs.io/en/latest/component/data.html#data-loader>`_ which is a component of `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ .
|
||||||
- If users require more complicated data processing (e.g. data normalization), `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ support user-customized processors to process data(some predefined processors can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/processor.py>`_). The processors are different from operators in expression engine. It is designed for some complicated data processing methods which is hard to supported in operators in expression engine.
|
- If users require more complicated data processing (e.g. data normalization), `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ support user-customized processors to process data(some predefined processors can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/processor.py>`_). The processors are different from operators in expression engine. It is designed for some complicated data processing methods which is hard to supported in operators in expression engine.
|
||||||
- At last, `Dataset <https://qlib.readthedocs.io/en/latest/component/data.html#dataset>`_ is responsible to prepare model-specific dataset from the processed data of Data Handler
|
- At last, `Dataset <https://qlib.readthedocs.io/en/latest/component/data.html#dataset>`_ is responsible to prepare model-specific dataset from the processed data of Data Handler
|
||||||
|
|
||||||
Data Preparation
|
Data Preparation
|
||||||
============================
|
================
|
||||||
|
|
||||||
Qlib Format Data
|
Qlib Format Data
|
||||||
------------------
|
----------------
|
||||||
|
|
||||||
We've specially designed a data structure to manage financial data, please refer to the `File storage design section in Qlib paper <https://arxiv.org/abs/2009.11189>`_ for detailed information.
|
We've specially designed a data structure to manage financial data, please refer to the `File storage design section in Qlib paper <https://arxiv.org/abs/2009.11189>`_ for detailed information.
|
||||||
Such data will be stored with filename suffix `.bin` (We'll call them `.bin` file, `.bin` format, or qlib format). `.bin` file is designed for scientific computing on finance data.
|
Such data will be stored with filename suffix `.bin` (We'll call them `.bin` file, `.bin` format, or qlib format). `.bin` file is designed for scientific computing on finance data.
|
||||||
@@ -50,9 +50,9 @@ Alpha158 √ √
|
|||||||
Also, ``Qlib`` provides a high-frequency dataset. Users can run a high-frequency dataset example through this `link <https://github.com/microsoft/qlib/tree/main/examples/highfreq>`_.
|
Also, ``Qlib`` provides a high-frequency dataset. Users can run a high-frequency dataset example through this `link <https://github.com/microsoft/qlib/tree/main/examples/highfreq>`_.
|
||||||
|
|
||||||
Qlib Format Dataset
|
Qlib Format Dataset
|
||||||
--------------------
|
-------------------
|
||||||
``Qlib`` has provided an off-the-shelf dataset in `.bin` format, users could use the script ``scripts/get_data.py`` to download the China-Stock dataset as follows.
|
``Qlib`` has provided an off-the-shelf dataset in `.bin` format, users could use the script ``scripts/get_data.py`` to download the China-Stock dataset as follows.
|
||||||
The price volume data look different from the actual dealling price because of they are **adjusted** (`adjusted price <https://www.investopedia.com/terms/a/adjusted_closing_price.asp>`_). And then you may find that the adjusted price may be different from different data sources. This is because different data sources may vary in the way of adjusting prices. Qlib normalize the price on first trading day of each stock to 1 when adjusting them.
|
The price volume data look different from the actual dealling price because of they are **adjusted** (`adjusted price <https://www.investopedia.com/terms/a/adjusted_closing_price.asp>`_). And then you may find that the adjusted price may be different from different data sources. This is because different data sources may vary in the way of adjusting prices. Qlib normalize the price on first trading day of each stock to 1 when adjusting them.
|
||||||
Users can leverage `$factor` to get the original trading price (e.g. `$close / $factor` to get the original close price).
|
Users can leverage `$factor` to get the original trading price (e.g. `$close / $factor` to get the original close price).
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
@@ -104,7 +104,7 @@ Automatic update of daily frequency data
|
|||||||
|
|
||||||
|
|
||||||
Converting CSV Format into Qlib Format
|
Converting CSV Format into Qlib Format
|
||||||
-------------------------------------------
|
--------------------------------------
|
||||||
|
|
||||||
``Qlib`` has provided the script ``scripts/dump_bin.py`` to convert **any** data in CSV format into `.bin` files (``Qlib`` format) as long as they are in the correct format.
|
``Qlib`` has provided the script ``scripts/dump_bin.py`` to convert **any** data in CSV format into `.bin` files (``Qlib`` format) as long as they are in the correct format.
|
||||||
|
|
||||||
@@ -126,16 +126,16 @@ Users can also provide their own data in CSV format. However, the CSV data **mus
|
|||||||
- CSV file is named after a specific stock *or* the CSV file includes a column of the stock name
|
- CSV file is named after a specific stock *or* the CSV file includes a column of the stock name
|
||||||
|
|
||||||
- Name the CSV file after a stock: `SH600000.csv`, `AAPL.csv` (not case sensitive).
|
- Name the CSV file after a stock: `SH600000.csv`, `AAPL.csv` (not case sensitive).
|
||||||
|
|
||||||
- CSV file includes a column of the stock name. User **must** specify the column name when dumping the data. Here is an example:
|
- CSV file includes a column of the stock name. User **must** specify the column name when dumping the data. Here is an example:
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
python scripts/dump_bin.py dump_all ... --symbol_field_name symbol
|
python scripts/dump_bin.py dump_all ... --symbol_field_name symbol
|
||||||
|
|
||||||
where the data are in the following format:
|
where the data are in the following format:
|
||||||
|
|
||||||
.. code-block::
|
.. code-block::
|
||||||
|
|
||||||
symbol,close
|
symbol,close
|
||||||
SH600000,120
|
SH600000,120
|
||||||
@@ -145,10 +145,10 @@ Users can also provide their own data in CSV format. However, the CSV data **mus
|
|||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
python scripts/dump_bin.py dump_all ... --date_field_name date
|
python scripts/dump_bin.py dump_all ... --date_field_name date
|
||||||
|
|
||||||
where the data are in the following format:
|
where the data are in the following format:
|
||||||
|
|
||||||
.. code-block::
|
.. code-block::
|
||||||
|
|
||||||
symbol,date,close,open,volume
|
symbol,date,close,open,volume
|
||||||
SH600000,2020-11-01,120,121,12300000
|
SH600000,2020-11-01,120,121,12300000
|
||||||
@@ -172,7 +172,7 @@ After conversion, users can find their Qlib format data in the directory `~/.qli
|
|||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
The arguments of `--include_fields` should correspond with the column names of CSV files. The columns names of dataset provided by ``Qlib`` should include open, close, high, low, volume and factor at least.
|
The arguments of `--include_fields` should correspond with the column names of CSV files. The columns names of dataset provided by ``Qlib`` should include open, close, high, low, volume and factor at least.
|
||||||
|
|
||||||
- `open`
|
- `open`
|
||||||
The adjusted opening price
|
The adjusted opening price
|
||||||
- `close`
|
- `close`
|
||||||
@@ -186,11 +186,11 @@ After conversion, users can find their Qlib format data in the directory `~/.qli
|
|||||||
- `factor`
|
- `factor`
|
||||||
The Restoration factor. Normally, ``factor = adjusted_price / original_price``, `adjusted price` reference: `split adjusted <https://www.investopedia.com/terms/s/splitadjusted.asp>`_
|
The Restoration factor. Normally, ``factor = adjusted_price / original_price``, `adjusted price` reference: `split adjusted <https://www.investopedia.com/terms/s/splitadjusted.asp>`_
|
||||||
|
|
||||||
In the convention of `Qlib` data processing, `open, close, high, low, volume, money and factor` will be set to NaN if the stock is suspended.
|
In the convention of `Qlib` data processing, `open, close, high, low, volume, money and factor` will be set to NaN if the stock is suspended.
|
||||||
If you want to use your own alpha-factor which can't be calculate by OCHLV, like PE, EPS and so on, you could add it to the CSV files with OHCLV together and then dump it to the Qlib format data.
|
If you want to use your own alpha-factor which can't be calculate by OCHLV, like PE, EPS and so on, you could add it to the CSV files with OHCLV together and then dump it to the Qlib format data.
|
||||||
|
|
||||||
Stock Pool (Market)
|
Stock Pool (Market)
|
||||||
--------------------------------
|
-------------------
|
||||||
|
|
||||||
``Qlib`` defines `stock pool <https://github.com/microsoft/qlib/blob/main/examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml#L4>`_ as stock list and their date ranges. Predefined stock pools (e.g. csi300) may be imported as follows.
|
``Qlib`` defines `stock pool <https://github.com/microsoft/qlib/blob/main/examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml#L4>`_ as stock list and their date ranges. Predefined stock pools (e.g. csi300) may be imported as follows.
|
||||||
|
|
||||||
@@ -200,7 +200,7 @@ Stock Pool (Market)
|
|||||||
|
|
||||||
|
|
||||||
Multiple Stock Modes
|
Multiple Stock Modes
|
||||||
--------------------------------
|
--------------------
|
||||||
|
|
||||||
``Qlib`` now provides two different stock modes for users: China-Stock Mode & US-Stock Mode. Here are some different settings of these two modes:
|
``Qlib`` now provides two different stock modes for users: China-Stock Mode & US-Stock Mode. Here are some different settings of these two modes:
|
||||||
|
|
||||||
@@ -218,23 +218,23 @@ The `trade unit` defines the unit number of stocks can be used in a trade, and t
|
|||||||
- Download china-stock in qlib format, please refer to section `Qlib Format Dataset <#qlib-format-dataset>`_.
|
- Download china-stock in qlib format, please refer to section `Qlib Format Dataset <#qlib-format-dataset>`_.
|
||||||
- Initialize ``Qlib`` in china-stock mode
|
- Initialize ``Qlib`` in china-stock mode
|
||||||
Supposed that users download their Qlib format data in the directory ``~/.qlib/qlib_data/cn_data``. Users only need to initialize ``Qlib`` as follows.
|
Supposed that users download their Qlib format data in the directory ``~/.qlib/qlib_data/cn_data``. Users only need to initialize ``Qlib`` as follows.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
from qlib.constant import REG_CN
|
from qlib.constant import REG_CN
|
||||||
qlib.init(provider_uri='~/.qlib/qlib_data/cn_data', region=REG_CN)
|
qlib.init(provider_uri='~/.qlib/qlib_data/cn_data', region=REG_CN)
|
||||||
|
|
||||||
|
|
||||||
- If users use ``Qlib`` in US-stock mode, US-stock data is required. ``Qlib`` also provides a script to download US-stock data. Users can use ``Qlib`` in US-stock mode according to the following steps:
|
- If users use ``Qlib`` in US-stock mode, US-stock data is required. ``Qlib`` also provides a script to download US-stock data. Users can use ``Qlib`` in US-stock mode according to the following steps:
|
||||||
- Download us-stock in qlib format, please refer to section `Qlib Format Dataset <#qlib-format-dataset>`_.
|
- Download us-stock in qlib format, please refer to section `Qlib Format Dataset <#qlib-format-dataset>`_.
|
||||||
- Initialize ``Qlib`` in US-stock mode
|
- Initialize ``Qlib`` in US-stock mode
|
||||||
Supposed that users prepare their Qlib format data in the directory ``~/.qlib/qlib_data/us_data``. Users only need to initialize ``Qlib`` as follows.
|
Supposed that users prepare their Qlib format data in the directory ``~/.qlib/qlib_data/us_data``. Users only need to initialize ``Qlib`` as follows.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
from qlib.config import REG_US
|
from qlib.config import REG_US
|
||||||
qlib.init(provider_uri='~/.qlib/qlib_data/us_data', region=REG_US)
|
qlib.init(provider_uri='~/.qlib/qlib_data/us_data', region=REG_US)
|
||||||
|
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
@@ -242,14 +242,14 @@ The `trade unit` defines the unit number of stocks can be used in a trade, and t
|
|||||||
|
|
||||||
|
|
||||||
Data API
|
Data API
|
||||||
========================
|
========
|
||||||
|
|
||||||
Data Retrieval
|
Data Retrieval
|
||||||
---------------
|
--------------
|
||||||
Users can use APIs in ``qlib.data`` to retrieve data, please refer to `Data Retrieval <../start/getdata.html>`_.
|
Users can use APIs in ``qlib.data`` to retrieve data, please refer to `Data Retrieval <../start/getdata.html>`_.
|
||||||
|
|
||||||
Feature
|
Feature
|
||||||
------------------
|
-------
|
||||||
|
|
||||||
``Qlib`` provides `Feature` and `ExpressionOps` to fetch the features according to users' needs.
|
``Qlib`` provides `Feature` and `ExpressionOps` to fetch the features according to users' needs.
|
||||||
|
|
||||||
@@ -264,7 +264,7 @@ Feature
|
|||||||
To know more about ``Feature``, please refer to `Feature API <../reference/api.html#module-qlib.data.base>`_.
|
To know more about ``Feature``, please refer to `Feature API <../reference/api.html#module-qlib.data.base>`_.
|
||||||
|
|
||||||
Filter
|
Filter
|
||||||
-------------------
|
------
|
||||||
``Qlib`` provides `NameDFilter` and `ExpressionDFilter` to filter the instruments according to users' needs.
|
``Qlib`` provides `NameDFilter` and `ExpressionDFilter` to filter the instruments according to users' needs.
|
||||||
|
|
||||||
- `NameDFilter`
|
- `NameDFilter`
|
||||||
@@ -272,7 +272,7 @@ Filter
|
|||||||
|
|
||||||
- `ExpressionDFilter`
|
- `ExpressionDFilter`
|
||||||
Expression dynamic instrument filter. Filter the instruments based on a certain expression. An expression rule indicating a certain feature field is required.
|
Expression dynamic instrument filter. Filter the instruments based on a certain expression. An expression rule indicating a certain feature field is required.
|
||||||
|
|
||||||
- `basic features filter`: rule_expression = '$close/$open>5'
|
- `basic features filter`: rule_expression = '$close/$open>5'
|
||||||
- `cross-sectional features filter` \: rule_expression = '$rank($close)<10'
|
- `cross-sectional features filter` \: rule_expression = '$rank($close)<10'
|
||||||
- `time-sequence features filter`: rule_expression = '$Ref($close, 3)>100'
|
- `time-sequence features filter`: rule_expression = '$Ref($close, 3)>100'
|
||||||
@@ -299,29 +299,29 @@ Here is a simple example showing how to use filter in a basic ``Qlib`` workflow
|
|||||||
To know more about ``Filter``, please refer to `Filter API <../reference/api.html#module-qlib.data.filter>`_.
|
To know more about ``Filter``, please refer to `Filter API <../reference/api.html#module-qlib.data.filter>`_.
|
||||||
|
|
||||||
Reference
|
Reference
|
||||||
-------------
|
---------
|
||||||
|
|
||||||
To know more about ``Data API``, please refer to `Data API <../reference/api.html#data>`_.
|
To know more about ``Data API``, please refer to `Data API <../reference/api.html#data>`_.
|
||||||
|
|
||||||
|
|
||||||
Data Loader
|
Data Loader
|
||||||
=================
|
===========
|
||||||
|
|
||||||
``Data Loader`` in ``Qlib`` is designed to load raw data from the original data source. It will be loaded and used in the ``Data Handler`` module.
|
``Data Loader`` in ``Qlib`` is designed to load raw data from the original data source. It will be loaded and used in the ``Data Handler`` module.
|
||||||
|
|
||||||
QlibDataLoader
|
QlibDataLoader
|
||||||
---------------
|
--------------
|
||||||
|
|
||||||
The ``QlibDataLoader`` class in ``Qlib`` is such an interface that allows users to load raw data from the ``Qlib`` data source.
|
The ``QlibDataLoader`` class in ``Qlib`` is such an interface that allows users to load raw data from the ``Qlib`` data source.
|
||||||
|
|
||||||
StaticDataLoader
|
StaticDataLoader
|
||||||
---------------
|
----------------
|
||||||
|
|
||||||
The ``StaticDataLoader`` class in ``Qlib`` is such an interface that allows users to load raw data from file or as provided.
|
The ``StaticDataLoader`` class in ``Qlib`` is such an interface that allows users to load raw data from file or as provided.
|
||||||
|
|
||||||
|
|
||||||
Interface
|
Interface
|
||||||
------------
|
---------
|
||||||
|
|
||||||
Here are some interfaces of the ``QlibDataLoader`` class:
|
Here are some interfaces of the ``QlibDataLoader`` class:
|
||||||
|
|
||||||
@@ -329,28 +329,28 @@ Here are some interfaces of the ``QlibDataLoader`` class:
|
|||||||
:members:
|
:members:
|
||||||
|
|
||||||
API
|
API
|
||||||
-----------
|
---
|
||||||
|
|
||||||
To know more about ``Data Loader``, please refer to `Data Loader API <../reference/api.html#module-qlib.data.dataset.loader>`_.
|
To know more about ``Data Loader``, please refer to `Data Loader API <../reference/api.html#module-qlib.data.dataset.loader>`_.
|
||||||
|
|
||||||
|
|
||||||
Data Handler
|
Data Handler
|
||||||
=================
|
============
|
||||||
|
|
||||||
The ``Data Handler`` module in ``Qlib`` is designed to handler those common data processing methods which will be used by most of the models.
|
The ``Data Handler`` module in ``Qlib`` is designed to handler those common data processing methods which will be used by most of the models.
|
||||||
|
|
||||||
Users can use ``Data Handler`` in an automatic workflow by ``qrun``, refer to `Workflow: Workflow Management <workflow.html>`_ for more details.
|
Users can use ``Data Handler`` in an automatic workflow by ``qrun``, refer to `Workflow: Workflow Management <workflow.html>`_ for more details.
|
||||||
|
|
||||||
DataHandlerLP
|
DataHandlerLP
|
||||||
--------------
|
-------------
|
||||||
|
|
||||||
In addition to use ``Data Handler`` in an automatic workflow with ``qrun``, ``Data Handler`` can be used as an independent module, by which users can easily preprocess data (standardization, remove NaN, etc.) and build datasets.
|
In addition to use ``Data Handler`` in an automatic workflow with ``qrun``, ``Data Handler`` can be used as an independent module, by which users can easily preprocess data (standardization, remove NaN, etc.) and build datasets.
|
||||||
|
|
||||||
In order to achieve so, ``Qlib`` provides a base class `qlib.data.dataset.DataHandlerLP <../reference/api.html#qlib.data.dataset.handler.DataHandlerLP>`_. The core idea of this class is that: we will have some learnable ``Processors`` which can learn the parameters of data processing(e.g., parameters for zscore normalization). When new data comes in, these `trained` ``Processors`` can then process the new data and thus processing real-time data in an efficient way becomes possible. More information about ``Processors`` will be listed in the next subsection.
|
In order to achieve so, ``Qlib`` provides a base class `qlib.data.dataset.DataHandlerLP <../reference/api.html#qlib.data.dataset.handler.DataHandlerLP>`_. The core idea of this class is that: we will have some learnable ``Processors`` which can learn the parameters of data processing(e.g., parameters for zscore normalization). When new data comes in, these `trained` ``Processors`` can then process the new data and thus processing real-time data in an efficient way becomes possible. More information about ``Processors`` will be listed in the next subsection.
|
||||||
|
|
||||||
|
|
||||||
Interface
|
Interface
|
||||||
----------------------
|
---------
|
||||||
|
|
||||||
Here are some important interfaces that ``DataHandlerLP`` provides:
|
Here are some important interfaces that ``DataHandlerLP`` provides:
|
||||||
|
|
||||||
@@ -364,7 +364,7 @@ Also, users can pass ``qlib.contrib.data.processor.ConfigSectionProcessor`` that
|
|||||||
|
|
||||||
|
|
||||||
Processor
|
Processor
|
||||||
----------
|
---------
|
||||||
|
|
||||||
The ``Processor`` module in ``Qlib`` is designed to be learnable and it is responsible for handling data processing such as `normalization` and `drop none/nan features/labels`.
|
The ``Processor`` module in ``Qlib`` is designed to be learnable and it is responsible for handling data processing such as `normalization` and `drop none/nan features/labels`.
|
||||||
|
|
||||||
@@ -382,14 +382,14 @@ The ``Processor`` module in ``Qlib`` is designed to be learnable and it is respo
|
|||||||
- ``CSRankNorm``: `processor` that applies cross sectional rank normalization.
|
- ``CSRankNorm``: `processor` that applies cross sectional rank normalization.
|
||||||
- ``CSZFillna``: `processor` that fills N/A values in a cross sectional way by the mean of the column.
|
- ``CSZFillna``: `processor` that fills N/A values in a cross sectional way by the mean of the column.
|
||||||
|
|
||||||
Users can also create their own `processor` by inheriting the base class of ``Processor``. Please refer to the implementation of all the processors for more information (`Processor Link <https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/processor.py>`_).
|
Users can also create their own `processor` by inheriting the base class of ``Processor``. Please refer to the implementation of all the processors for more information (`Processor Link <https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/processor.py>`_).
|
||||||
|
|
||||||
To know more about ``Processor``, please refer to `Processor API <../reference/api.html#module-qlib.data.dataset.processor>`_.
|
To know more about ``Processor``, please refer to `Processor API <../reference/api.html#module-qlib.data.dataset.processor>`_.
|
||||||
|
|
||||||
Example
|
Example
|
||||||
--------------
|
-------
|
||||||
|
|
||||||
``Data Handler`` can be run with ``qrun`` by modifying the configuration file, and can also be used as a single module.
|
``Data Handler`` can be run with ``qrun`` by modifying the configuration file, and can also be used as a single module.
|
||||||
|
|
||||||
Know more about how to run ``Data Handler`` with ``qrun``, please refer to `Workflow: Workflow Management <workflow.html>`_
|
Know more about how to run ``Data Handler`` with ``qrun``, please refer to `Workflow: Workflow Management <workflow.html>`_
|
||||||
|
|
||||||
@@ -427,17 +427,17 @@ Qlib provides implemented data handler `Alpha158`. The following example shows h
|
|||||||
.. note:: In the ``Alpha158``, ``Qlib`` uses the label `Ref($close, -2)/Ref($close, -1) - 1` that means the change from T+1 to T+2, rather than `Ref($close, -1)/$close - 1`, of which the reason is that when getting the T day close price of a china stock, the stock can be bought on T+1 day and sold on T+2 day.
|
.. note:: In the ``Alpha158``, ``Qlib`` uses the label `Ref($close, -2)/Ref($close, -1) - 1` that means the change from T+1 to T+2, rather than `Ref($close, -1)/$close - 1`, of which the reason is that when getting the T day close price of a china stock, the stock can be bought on T+1 day and sold on T+2 day.
|
||||||
|
|
||||||
API
|
API
|
||||||
---------
|
---
|
||||||
|
|
||||||
To know more about ``Data Handler``, please refer to `Data Handler API <../reference/api.html#module-qlib.data.dataset.handler>`_.
|
To know more about ``Data Handler``, please refer to `Data Handler API <../reference/api.html#module-qlib.data.dataset.handler>`_.
|
||||||
|
|
||||||
|
|
||||||
Dataset
|
Dataset
|
||||||
=================
|
=======
|
||||||
|
|
||||||
The ``Dataset`` module in ``Qlib`` aims to prepare data for model training and inferencing.
|
The ``Dataset`` module in ``Qlib`` aims to prepare data for model training and inferencing.
|
||||||
|
|
||||||
The motivation of this module is that we want to maximize the flexibility of different models to handle data that are suitable for themselves. This module gives the model the flexibility to process their data in an unique way. For instance, models such as ``GBDT`` may work well on data that contains `nan` or `None` value, while neural networks such as ``MLP`` will break down on such data.
|
The motivation of this module is that we want to maximize the flexibility of different models to handle data that are suitable for themselves. This module gives the model the flexibility to process their data in an unique way. For instance, models such as ``GBDT`` may work well on data that contains `nan` or `None` value, while neural networks such as ``MLP`` will break down on such data.
|
||||||
|
|
||||||
If user's model need process its data in a different way, user could implement his own ``Dataset`` class. If the model's
|
If user's model need process its data in a different way, user could implement his own ``Dataset`` class. If the model's
|
||||||
data processing is not special, ``DatasetH`` can be used directly.
|
data processing is not special, ``DatasetH`` can be used directly.
|
||||||
@@ -448,18 +448,18 @@ The ``DatasetH`` class is the `dataset` with `Data Handler`. Here is the most im
|
|||||||
:members:
|
:members:
|
||||||
|
|
||||||
API
|
API
|
||||||
---------
|
---
|
||||||
|
|
||||||
To know more about ``Dataset``, please refer to `Dataset API <../reference/api.html#dataset>`_.
|
To know more about ``Dataset``, please refer to `Dataset API <../reference/api.html#dataset>`_.
|
||||||
|
|
||||||
|
|
||||||
Cache
|
Cache
|
||||||
==========
|
=====
|
||||||
|
|
||||||
``Cache`` is an optional module that helps accelerate providing data by saving some frequently-used data as cache file. ``Qlib`` provides a `Memcache` class to cache the most-frequently-used data in memory, an inheritable `ExpressionCache` class, and an inheritable `DatasetCache` class.
|
``Cache`` is an optional module that helps accelerate providing data by saving some frequently-used data as cache file. ``Qlib`` provides a `Memcache` class to cache the most-frequently-used data in memory, an inheritable `ExpressionCache` class, and an inheritable `DatasetCache` class.
|
||||||
|
|
||||||
Global Memory Cache
|
Global Memory Cache
|
||||||
---------------------
|
-------------------
|
||||||
|
|
||||||
`Memcache` is a global memory cache mechanism that composes of three `MemCacheUnit` instances to cache **Calendar**, **Instruments**, and **Features**. The `MemCache` is defined globally in `cache.py` as `H`. Users can use `H['c'], H['i'], H['f']` to get/set `memcache`.
|
`Memcache` is a global memory cache mechanism that composes of three `MemCacheUnit` instances to cache **Calendar**, **Instruments**, and **Features**. The `MemCache` is defined globally in `cache.py` as `H`. Users can use `H['c'], H['i'], H['f']` to get/set `memcache`.
|
||||||
|
|
||||||
@@ -471,7 +471,7 @@ Global Memory Cache
|
|||||||
|
|
||||||
|
|
||||||
ExpressionCache
|
ExpressionCache
|
||||||
-----------------
|
---------------
|
||||||
|
|
||||||
`ExpressionCache` is a cache mechanism that saves expressions such as **Mean($close, 5)**. Users can inherit this base class to define their own cache mechanism that saves expressions according to the following steps.
|
`ExpressionCache` is a cache mechanism that saves expressions such as **Mean($close, 5)**. Users can inherit this base class to define their own cache mechanism that saves expressions according to the following steps.
|
||||||
|
|
||||||
@@ -486,7 +486,7 @@ The following shows the details about the interfaces:
|
|||||||
``Qlib`` has currently provided implemented disk cache `DiskExpressionCache` which inherits from `ExpressionCache` . The expressions data will be stored in the disk.
|
``Qlib`` has currently provided implemented disk cache `DiskExpressionCache` which inherits from `ExpressionCache` . The expressions data will be stored in the disk.
|
||||||
|
|
||||||
DatasetCache
|
DatasetCache
|
||||||
-----------------
|
------------
|
||||||
|
|
||||||
`DatasetCache` is a cache mechanism that saves datasets. A certain dataset is regulated by a stock pool configuration (or a series of instruments, though not recommended), a list of expressions or static feature fields, the start time, and end time for the collected features and the frequency. Users can inherit this base class to define their own cache mechanism that saves datasets according to the following steps.
|
`DatasetCache` is a cache mechanism that saves datasets. A certain dataset is regulated by a stock pool configuration (or a series of instruments, though not recommended), a list of expressions or static feature fields, the start time, and end time for the collected features and the frequency. Users can inherit this base class to define their own cache mechanism that saves datasets according to the following steps.
|
||||||
|
|
||||||
@@ -503,7 +503,7 @@ The following shows the details about the interfaces:
|
|||||||
|
|
||||||
|
|
||||||
Data and Cache File Structure
|
Data and Cache File Structure
|
||||||
==================================
|
=============================
|
||||||
|
|
||||||
We've specially designed a file structure to manage data and cache, please refer to the `File storage design section in Qlib paper <https://arxiv.org/abs/2009.11189>`_ for detailed information. The file structure of data and cache is listed as follows.
|
We've specially designed a file structure to manage data and cache, please refer to the `File storage design section in Qlib paper <https://arxiv.org/abs/2009.11189>`_ for detailed information. The file structure of data and cache is listed as follows.
|
||||||
|
|
||||||
@@ -536,4 +536,3 @@ We've specially designed a file structure to manage data and cache, please refer
|
|||||||
- .meta : an assorted meta file recording the stockpool config, field names and visit times
|
- .meta : an assorted meta file recording the stockpool config, field names and visit times
|
||||||
- .index : an assorted index file recording the line index of all calendars
|
- .index : an assorted index file recording the line index of all calendars
|
||||||
- ...
|
- ...
|
||||||
|
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
.. _highfreq:
|
.. _highfreq:
|
||||||
|
|
||||||
============================================
|
========================================================================
|
||||||
Design of Nested Decision Execution Framework for High-Frequency Trading
|
Design of Nested Decision Execution Framework for High-Frequency Trading
|
||||||
============================================
|
========================================================================
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
|
|
||||||
Daily trading (e.g. portfolio management) and intraday trading (e.g. orders execution) are two hot topics in Quant investment and usually studied separately.
|
Daily trading (e.g. portfolio management) and intraday trading (e.g. orders execution) are two hot topics in Quant investment and usually studied separately.
|
||||||
|
|
||||||
@@ -15,18 +15,18 @@ In order to support the joint backtest strategies in multiple levels, a correspo
|
|||||||
|
|
||||||
Besides backtesting, the optimization of strategies from different levels is not standalone and can be affected by each other.
|
Besides backtesting, the optimization of strategies from different levels is not standalone and can be affected by each other.
|
||||||
For example, the best portfolio management strategy may change with the performance of order executions(e.g. a portfolio with higher turnover may becomes a better choice when we improve the order execution strategies).
|
For example, the best portfolio management strategy may change with the performance of order executions(e.g. a portfolio with higher turnover may becomes a better choice when we improve the order execution strategies).
|
||||||
To achieve the overall good performance , it is necessary to consider the interaction of strategies in different level.
|
To achieve the overall good performance , it is necessary to consider the interaction of strategies in different level.
|
||||||
|
|
||||||
Therefore, building a new framework for trading in multiple levels becomes necessary to solve the various problems mentioned above, for which we designed a nested decision execution framework that consider the interaction of strategies.
|
Therefore, building a new framework for trading in multiple levels becomes necessary to solve the various problems mentioned above, for which we designed a nested decision execution framework that consider the interaction of strategies.
|
||||||
|
|
||||||
.. image:: ../_static/img/framework.svg
|
.. image:: ../_static/img/framework.svg
|
||||||
|
|
||||||
The design of the framework is shown in the yellow part in the middle of the figure above. Each level consists of ``Trading Agent`` and ``Execution Env``. ``Trading Agent`` has its own data processing module (``Information Extractor``), forecasting module (``Forecast Model``) and decision generator (``Decision Generator``). The trading algorithm generates the decisions by the ``Decision Generator`` based on the forecast signals output by the ``Forecast Module``, and the decisions generated by the trading algorithm are passed to the ``Execution Env``, which returns the execution results.
|
The design of the framework is shown in the yellow part in the middle of the figure above. Each level consists of ``Trading Agent`` and ``Execution Env``. ``Trading Agent`` has its own data processing module (``Information Extractor``), forecasting module (``Forecast Model``) and decision generator (``Decision Generator``). The trading algorithm generates the decisions by the ``Decision Generator`` based on the forecast signals output by the ``Forecast Module``, and the decisions generated by the trading algorithm are passed to the ``Execution Env``, which returns the execution results.
|
||||||
|
|
||||||
The frequency of trading algorithm, decision content and execution environment can be customized by users (e.g. intraday trading, daily-frequency trading, weekly-frequency trading), and the execution environment can be nested with finer-grained trading algorithm and execution environment inside (i.e. sub-workflow in the figure, e.g. daily-frequency orders can be turned into finer-grained decisions by splitting orders within the day). The flexibility of nested decision execution framework makes it easy for users to explore the effects of combining different levels of trading strategies and break down the optimization barriers between different levels of trading algorithm.
|
The frequency of trading algorithm, decision content and execution environment can be customized by users (e.g. intraday trading, daily-frequency trading, weekly-frequency trading), and the execution environment can be nested with finer-grained trading algorithm and execution environment inside (i.e. sub-workflow in the figure, e.g. daily-frequency orders can be turned into finer-grained decisions by splitting orders within the day). The flexibility of nested decision execution framework makes it easy for users to explore the effects of combining different levels of trading strategies and break down the optimization barriers between different levels of trading algorithm.
|
||||||
|
|
||||||
Example
|
Example
|
||||||
===========================
|
=======
|
||||||
|
|
||||||
An example of nested decision execution framework for high-frequency can be found `here <https://github.com/microsoft/qlib/blob/main/examples/nested_decision_execution/workflow.py>`_.
|
An example of nested decision execution framework for high-frequency can be found `here <https://github.com/microsoft/qlib/blob/main/examples/nested_decision_execution/workflow.py>`_.
|
||||||
|
|
||||||
|
|||||||
@@ -1,17 +1,17 @@
|
|||||||
.. _meta:
|
.. _meta:
|
||||||
|
|
||||||
=================================
|
======================================================
|
||||||
Meta Controller: Meta-Task & Meta-Dataset & Meta-Model
|
Meta Controller: Meta-Task & Meta-Dataset & Meta-Model
|
||||||
=================================
|
======================================================
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
=============
|
============
|
||||||
``Meta Controller`` provides guidance to ``Forecast Model``, which aims to learn regular patterns among a series of forecasting tasks and use learned patterns to guide forthcoming forecasting tasks. Users can implement their own meta-model instance based on ``Meta Controller`` module.
|
``Meta Controller`` provides guidance to ``Forecast Model``, which aims to learn regular patterns among a series of forecasting tasks and use learned patterns to guide forthcoming forecasting tasks. Users can implement their own meta-model instance based on ``Meta Controller`` module.
|
||||||
|
|
||||||
Meta Task
|
Meta Task
|
||||||
=============
|
=========
|
||||||
|
|
||||||
A `Meta Task` instance is the basic element in the meta-learning framework. It saves the data that can be used for the `Meta Model`. Multiple `Meta Task` instances may share the same `Data Handler`, controlled by `Meta Dataset`. Users should use `prepare_task_data()` to obtain the data that can be directly fed into the `Meta Model`.
|
A `Meta Task` instance is the basic element in the meta-learning framework. It saves the data that can be used for the `Meta Model`. Multiple `Meta Task` instances may share the same `Data Handler`, controlled by `Meta Dataset`. Users should use `prepare_task_data()` to obtain the data that can be directly fed into the `Meta Model`.
|
||||||
|
|
||||||
@@ -19,7 +19,7 @@ A `Meta Task` instance is the basic element in the meta-learning framework. It s
|
|||||||
:members:
|
:members:
|
||||||
|
|
||||||
Meta Dataset
|
Meta Dataset
|
||||||
=============
|
============
|
||||||
|
|
||||||
`Meta Dataset` controls the meta-information generating process. It is on the duty of providing data for training the `Meta Model`. Users should use `prepare_tasks` to retrieve a list of `Meta Task` instances.
|
`Meta Dataset` controls the meta-information generating process. It is on the duty of providing data for training the `Meta Model`. Users should use `prepare_tasks` to retrieve a list of `Meta Task` instances.
|
||||||
|
|
||||||
@@ -27,26 +27,26 @@ Meta Dataset
|
|||||||
:members:
|
:members:
|
||||||
|
|
||||||
Meta Model
|
Meta Model
|
||||||
=============
|
==========
|
||||||
|
|
||||||
General Meta Model
|
General Meta Model
|
||||||
------------------
|
------------------
|
||||||
`Meta Model` instance is the part that controls the workflow. The usage of the `Meta Model` includes:
|
`Meta Model` instance is the part that controls the workflow. The usage of the `Meta Model` includes:
|
||||||
1. Users train their `Meta Model` with the `fit` function.
|
1. Users train their `Meta Model` with the `fit` function.
|
||||||
2. The `Meta Model` instance guides the workflow by giving useful information via the `inference` function.
|
2. The `Meta Model` instance guides the workflow by giving useful information via the `inference` function.
|
||||||
|
|
||||||
.. autoclass:: qlib.model.meta.model.MetaModel
|
.. autoclass:: qlib.model.meta.model.MetaModel
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Meta Task Model
|
Meta Task Model
|
||||||
------------------
|
---------------
|
||||||
This type of meta-model may interact with task definitions directly. Then, the `Meta Task Model` is the class for them to inherit from. They guide the base tasks by modifying the base task definitions. The function `prepare_tasks` can be used to obtain the modified base task definitions.
|
This type of meta-model may interact with task definitions directly. Then, the `Meta Task Model` is the class for them to inherit from. They guide the base tasks by modifying the base task definitions. The function `prepare_tasks` can be used to obtain the modified base task definitions.
|
||||||
|
|
||||||
.. autoclass:: qlib.model.meta.model.MetaTaskModel
|
.. autoclass:: qlib.model.meta.model.MetaTaskModel
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Meta Guide Model
|
Meta Guide Model
|
||||||
------------------
|
----------------
|
||||||
This type of meta-model participates in the training process of the base forecasting model. The meta-model may guide the base forecasting models during their training to improve their performances.
|
This type of meta-model participates in the training process of the base forecasting model. The meta-model may guide the base forecasting models during their training to improve their performances.
|
||||||
|
|
||||||
.. autoclass:: qlib.model.meta.model.MetaGuideModel
|
.. autoclass:: qlib.model.meta.model.MetaGuideModel
|
||||||
@@ -54,9 +54,9 @@ This type of meta-model participates in the training process of the base forecas
|
|||||||
|
|
||||||
|
|
||||||
Example
|
Example
|
||||||
=============
|
=======
|
||||||
``Qlib`` provides an implementation of ``Meta Model`` module, ``DDG-DA``,
|
``Qlib`` provides an implementation of ``Meta Model`` module, ``DDG-DA``,
|
||||||
which adapts to the market dynamics.
|
which adapts to the market dynamics.
|
||||||
|
|
||||||
``DDG-DA`` includes four steps:
|
``DDG-DA`` includes four steps:
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +1,13 @@
|
|||||||
.. _model:
|
.. _model:
|
||||||
|
|
||||||
============================================
|
===========================================
|
||||||
Forecast Model: Model Training & Prediction
|
Forecast Model: Model Training & Prediction
|
||||||
============================================
|
===========================================
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
|
|
||||||
``Forecast Model`` is designed to make the `prediction score` about stocks. Users can use the ``Forecast Model`` in an automatic workflow by ``qrun``, please refer to `Workflow: Workflow Management <workflow.html>`_.
|
``Forecast Model`` is designed to make the `prediction score` about stocks. Users can use the ``Forecast Model`` in an automatic workflow by ``qrun``, please refer to `Workflow: Workflow Management <workflow.html>`_.
|
||||||
|
|
||||||
Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Forecast Model`` can be used as an independent module also.
|
Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Forecast Model`` can be used as an independent module also.
|
||||||
|
|
||||||
@@ -22,11 +22,11 @@ The base class provides the following interfaces:
|
|||||||
:members:
|
:members:
|
||||||
|
|
||||||
``Qlib`` also provides a base class `qlib.model.base.ModelFT <../reference/api.html#qlib.model.base.ModelFT>`_, which includes the method for finetuning the model.
|
``Qlib`` also provides a base class `qlib.model.base.ModelFT <../reference/api.html#qlib.model.base.ModelFT>`_, which includes the method for finetuning the model.
|
||||||
|
|
||||||
For other interfaces such as `finetune`, please refer to `Model API <../reference/api.html#module-qlib.model.base>`_.
|
For other interfaces such as `finetune`, please refer to `Model API <../reference/api.html#module-qlib.model.base>`_.
|
||||||
|
|
||||||
Example
|
Example
|
||||||
==================
|
=======
|
||||||
|
|
||||||
``Qlib``'s `Model Zoo` includes models such as ``LightGBM``, ``MLP``, ``LSTM``, etc.. These models are treated as the baselines of ``Forecast Model``. The following steps show how to run`` LightGBM`` as an independent module.
|
``Qlib``'s `Model Zoo` includes models such as ``LightGBM``, ``MLP``, ``LSTM``, etc.. These models are treated as the baselines of ``Forecast Model``. The following steps show how to run`` LightGBM`` as an independent module.
|
||||||
|
|
||||||
@@ -84,7 +84,7 @@ Example
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
# model initiaiton
|
# model initiaiton
|
||||||
model = init_instance_by_config(task["model"])
|
model = init_instance_by_config(task["model"])
|
||||||
dataset = init_instance_by_config(task["dataset"])
|
dataset = init_instance_by_config(task["dataset"])
|
||||||
@@ -100,22 +100,22 @@ Example
|
|||||||
sr = SignalRecord(model, dataset, recorder)
|
sr = SignalRecord(model, dataset, recorder)
|
||||||
sr.generate()
|
sr.generate()
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
`Alpha158` is the data handler provided by ``Qlib``, please refer to `Data Handler <data.html#data-handler>`_.
|
`Alpha158` is the data handler provided by ``Qlib``, please refer to `Data Handler <data.html#data-handler>`_.
|
||||||
`SignalRecord` is the `Record Template` in ``Qlib``, please refer to `Workflow <recorder.html#record-template>`_.
|
`SignalRecord` is the `Record Template` in ``Qlib``, please refer to `Workflow <recorder.html#record-template>`_.
|
||||||
|
|
||||||
Also, the above example has been given in ``examples/train_backtest_analyze.ipynb``.
|
Also, the above example has been given in ``examples/train_backtest_analyze.ipynb``.
|
||||||
Technically, the meaning of the model prediction depends on the label setting designed by user.
|
Technically, the meaning of the model prediction depends on the label setting designed by user.
|
||||||
By default, the meaning of the score is normally the rating of the instruments by the forecasting model. The higher the score, the more profit the instruments.
|
By default, the meaning of the score is normally the rating of the instruments by the forecasting model. The higher the score, the more profit the instruments.
|
||||||
|
|
||||||
|
|
||||||
Custom Model
|
Custom Model
|
||||||
===================
|
============
|
||||||
|
|
||||||
Qlib supports custom models. If users are interested in customizing their own models and integrating the models into ``Qlib``, please refer to `Custom Model Integration <../start/integration.html>`_.
|
Qlib supports custom models. If users are interested in customizing their own models and integrating the models into ``Qlib``, please refer to `Custom Model Integration <../start/integration.html>`_.
|
||||||
|
|
||||||
|
|
||||||
API
|
API
|
||||||
===================
|
===
|
||||||
Please refer to `Model API <../reference/api.html#module-qlib.model.base>`_.
|
Please refer to `Model API <../reference/api.html#module-qlib.model.base>`_.
|
||||||
|
|||||||
@@ -1,13 +1,13 @@
|
|||||||
.. _online:
|
.. _online:
|
||||||
|
|
||||||
=================================
|
==============
|
||||||
Online Serving
|
Online Serving
|
||||||
=================================
|
==============
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
=============
|
============
|
||||||
|
|
||||||
.. image:: ../_static/img/online_serving.png
|
.. image:: ../_static/img/online_serving.png
|
||||||
:align: center
|
:align: center
|
||||||
@@ -15,7 +15,7 @@ Introduction
|
|||||||
|
|
||||||
In addition to backtesting, one way to test a model is effective is to make predictions in real market conditions or even do real trading based on those predictions.
|
In addition to backtesting, one way to test a model is effective is to make predictions in real market conditions or even do real trading based on those predictions.
|
||||||
``Online Serving`` is a set of modules for online models using the latest data,
|
``Online Serving`` is a set of modules for online models using the latest data,
|
||||||
which including `Online Manager <#Online Manager>`_, `Online Strategy <#Online Strategy>`_, `Online Tool <#Online Tool>`_, `Updater <#Updater>`_.
|
which including `Online Manager <#Online Manager>`_, `Online Strategy <#Online Strategy>`_, `Online Tool <#Online Tool>`_, `Updater <#Updater>`_.
|
||||||
|
|
||||||
`Here <https://github.com/microsoft/qlib/tree/main/examples/online_srv>`_ are several examples for reference, which demonstrate different features of ``Online Serving``.
|
`Here <https://github.com/microsoft/qlib/tree/main/examples/online_srv>`_ are several examples for reference, which demonstrate different features of ``Online Serving``.
|
||||||
If you have many models or `task` needs to be managed, please consider `Task Management <../advanced/task_management.html>`_.
|
If you have many models or `task` needs to be managed, please consider `Task Management <../advanced/task_management.html>`_.
|
||||||
@@ -28,25 +28,25 @@ Known limitations currently
|
|||||||
|
|
||||||
|
|
||||||
Online Manager
|
Online Manager
|
||||||
=============
|
==============
|
||||||
|
|
||||||
.. automodule:: qlib.workflow.online.manager
|
.. automodule:: qlib.workflow.online.manager
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Online Strategy
|
Online Strategy
|
||||||
=============
|
===============
|
||||||
|
|
||||||
.. automodule:: qlib.workflow.online.strategy
|
.. automodule:: qlib.workflow.online.strategy
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Online Tool
|
Online Tool
|
||||||
=============
|
===========
|
||||||
|
|
||||||
.. automodule:: qlib.workflow.online.utils
|
.. automodule:: qlib.workflow.online.utils
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Updater
|
Updater
|
||||||
=============
|
=======
|
||||||
|
|
||||||
.. automodule:: qlib.workflow.online.update
|
.. automodule:: qlib.workflow.online.update
|
||||||
:members:
|
:members:
|
||||||
|
|||||||
@@ -6,8 +6,8 @@ Qlib Recorder: Experiment Management
|
|||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
``Qlib`` contains an experiment management system named ``QlibRecorder``, which is designed to help users handle experiment and analyse results in an efficient way.
|
``Qlib`` contains an experiment management system named ``QlibRecorder``, which is designed to help users handle experiment and analyse results in an efficient way.
|
||||||
|
|
||||||
There are three components of the system:
|
There are three components of the system:
|
||||||
|
|
||||||
@@ -34,13 +34,13 @@ Here is a general view of the structure of the system:
|
|||||||
- Recorder 2
|
- Recorder 2
|
||||||
- ...
|
- ...
|
||||||
- ...
|
- ...
|
||||||
|
|
||||||
This experiment management system defines a set of interface and provided a concrete implementation ``MLflowExpManager``, which is based on the machine learning platform: ``MLFlow`` (`link <https://mlflow.org/>`_).
|
This experiment management system defines a set of interface and provided a concrete implementation ``MLflowExpManager``, which is based on the machine learning platform: ``MLFlow`` (`link <https://mlflow.org/>`_).
|
||||||
|
|
||||||
If users set the implementation of ``ExpManager`` to be ``MLflowExpManager``, they can use the command `mlflow ui` to visualize and check the experiment results. For more information, please refer to the related documents `here <https://www.mlflow.org/docs/latest/cli.html#mlflow-ui>`_.
|
If users set the implementation of ``ExpManager`` to be ``MLflowExpManager``, they can use the command `mlflow ui` to visualize and check the experiment results. For more information, please refer to the related documents `here <https://www.mlflow.org/docs/latest/cli.html#mlflow-ui>`_.
|
||||||
|
|
||||||
Qlib Recorder
|
Qlib Recorder
|
||||||
===================
|
=============
|
||||||
``QlibRecorder`` provides a high level API for users to use the experiment management system. The interfaces are wrapped in the variable ``R`` in ``Qlib``, and users can directly use ``R`` to interact with the system. The following command shows how to import ``R`` in Python:
|
``QlibRecorder`` provides a high level API for users to use the experiment management system. The interfaces are wrapped in the variable ``R`` in ``Qlib``, and users can directly use ``R`` to interact with the system. The following command shows how to import ``R`` in Python:
|
||||||
|
|
||||||
.. code-block:: Python
|
.. code-block:: Python
|
||||||
@@ -55,7 +55,7 @@ Here are the available interfaces of ``QlibRecorder``:
|
|||||||
:members:
|
:members:
|
||||||
|
|
||||||
Experiment Manager
|
Experiment Manager
|
||||||
===================
|
==================
|
||||||
|
|
||||||
The ``ExpManager`` module in ``Qlib`` is responsible for managing different experiments. Most of the APIs of ``ExpManager`` are similar to ``QlibRecorder``, and the most important API will be the ``get_exp`` method. User can directly refer to the documents above for some detailed information about how to use the ``get_exp`` method.
|
The ``ExpManager`` module in ``Qlib`` is responsible for managing different experiments. Most of the APIs of ``ExpManager`` are similar to ``QlibRecorder``, and the most important API will be the ``get_exp`` method. User can directly refer to the documents above for some detailed information about how to use the ``get_exp`` method.
|
||||||
|
|
||||||
@@ -65,7 +65,7 @@ The ``ExpManager`` module in ``Qlib`` is responsible for managing different expe
|
|||||||
For other interfaces such as `create_exp`, `delete_exp`, please refer to `Experiment Manager API <../reference/api.html#experiment-manager>`_.
|
For other interfaces such as `create_exp`, `delete_exp`, please refer to `Experiment Manager API <../reference/api.html#experiment-manager>`_.
|
||||||
|
|
||||||
Experiment
|
Experiment
|
||||||
===================
|
==========
|
||||||
|
|
||||||
The ``Experiment`` class is solely responsible for a single experiment, and it will handle any operations that are related to an experiment. Basic methods such as `start`, `end` an experiment are included. Besides, methods related to `recorders` are also available: such methods include `get_recorder` and `list_recorders`.
|
The ``Experiment`` class is solely responsible for a single experiment, and it will handle any operations that are related to an experiment. Basic methods such as `start`, `end` an experiment are included. Besides, methods related to `recorders` are also available: such methods include `get_recorder` and `list_recorders`.
|
||||||
|
|
||||||
@@ -77,7 +77,7 @@ For other interfaces such as `search_records`, `delete_recorder`, please refer t
|
|||||||
``Qlib`` also provides a default ``Experiment``, which will be created and used under certain situations when users use the APIs such as `log_metrics` or `get_exp`. If the default ``Experiment`` is used, there will be related logged information when running ``Qlib``. Users are able to change the name of the default ``Experiment`` in the config file of ``Qlib`` or during ``Qlib``'s `initialization <../start/initialization.html#parameters>`_, which is set to be '`Experiment`'.
|
``Qlib`` also provides a default ``Experiment``, which will be created and used under certain situations when users use the APIs such as `log_metrics` or `get_exp`. If the default ``Experiment`` is used, there will be related logged information when running ``Qlib``. Users are able to change the name of the default ``Experiment`` in the config file of ``Qlib`` or during ``Qlib``'s `initialization <../start/initialization.html#parameters>`_, which is set to be '`Experiment`'.
|
||||||
|
|
||||||
Recorder
|
Recorder
|
||||||
===================
|
========
|
||||||
|
|
||||||
The ``Recorder`` class is responsible for a single recorder. It will handle some detailed operations such as ``log_metrics``, ``log_params`` of a single run. It is designed to help user to easily track results and things being generated during a run.
|
The ``Recorder`` class is responsible for a single recorder. It will handle some detailed operations such as ``log_metrics``, ``log_params`` of a single run. It is designed to help user to easily track results and things being generated during a run.
|
||||||
|
|
||||||
@@ -89,7 +89,7 @@ Here are some important APIs that are not included in the ``QlibRecorder``:
|
|||||||
For other interfaces such as `save_objects`, `load_object`, please refer to `Recorder API <../reference/api.html#recorder>`_.
|
For other interfaces such as `save_objects`, `load_object`, please refer to `Recorder API <../reference/api.html#recorder>`_.
|
||||||
|
|
||||||
Record Template
|
Record Template
|
||||||
===================
|
===============
|
||||||
|
|
||||||
The ``RecordTemp`` class is a class that enables generate experiment results such as IC and backtest in a certain format. We have provided three different `Record Template` class:
|
The ``RecordTemp`` class is a class that enables generate experiment results such as IC and backtest in a certain format. We have provided three different `Record Template` class:
|
||||||
|
|
||||||
@@ -131,7 +131,7 @@ Here is a simple exampke of what is done in ``PortAnaRecord``, which users can r
|
|||||||
"close_cost": 0.0015,
|
"close_cost": 0.0015,
|
||||||
"min_cost": 5,
|
"min_cost": 5,
|
||||||
}
|
}
|
||||||
|
|
||||||
strategy = TopkDropoutStrategy(**STRATEGY_CONFIG)
|
strategy = TopkDropoutStrategy(**STRATEGY_CONFIG)
|
||||||
report_normal, positions_normal = normal_backtest(pred_score, strategy=strategy, **BACKTEST_CONFIG)
|
report_normal, positions_normal = normal_backtest(pred_score, strategy=strategy, **BACKTEST_CONFIG)
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
.. _report:
|
.. _report:
|
||||||
|
|
||||||
==========================================
|
=======================================
|
||||||
Analysis: Evaluation & Results Analysis
|
Analysis: Evaluation & Results Analysis
|
||||||
==========================================
|
=======================================
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
|
|
||||||
``Analysis`` is designed to show the graphical reports of ``Intraday Trading`` , which helps users to evaluate and analyse investment portfolios visually. The following are some graphics to view:
|
``Analysis`` is designed to show the graphical reports of ``Intraday Trading`` , which helps users to evaluate and analyse investment portfolios visually. The following are some graphics to view:
|
||||||
|
|
||||||
@@ -24,7 +24,7 @@ All of the accumulated profit metrics(e.g. return, max drawdown) in Qlib are cal
|
|||||||
This avoids the metrics or the plots being skewed exponentially over time.
|
This avoids the metrics or the plots being skewed exponentially over time.
|
||||||
|
|
||||||
Graphical Reports
|
Graphical Reports
|
||||||
===================
|
=================
|
||||||
|
|
||||||
Users can run the following code to get all supported reports.
|
Users can run the following code to get all supported reports.
|
||||||
|
|
||||||
@@ -41,13 +41,13 @@ Users can run the following code to get all supported reports.
|
|||||||
|
|
||||||
|
|
||||||
Usage & Example
|
Usage & Example
|
||||||
===================
|
===============
|
||||||
|
|
||||||
Usage of `analysis_position.report`
|
Usage of `analysis_position.report`
|
||||||
-----------------------------------
|
-----------------------------------
|
||||||
|
|
||||||
API
|
API
|
||||||
~~~~~~~~~~~~~~~~
|
~~~
|
||||||
|
|
||||||
.. automodule:: qlib.contrib.report.analysis_position.report
|
.. automodule:: qlib.contrib.report.analysis_position.report
|
||||||
:members:
|
:members:
|
||||||
@@ -58,7 +58,7 @@ Graphical Result
|
|||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
- Axis X: Trading day
|
- Axis X: Trading day
|
||||||
- Axis Y:
|
- Axis Y:
|
||||||
- `cum bench`
|
- `cum bench`
|
||||||
Cumulative returns series of benchmark
|
Cumulative returns series of benchmark
|
||||||
- `cum return wo cost`
|
- `cum return wo cost`
|
||||||
@@ -82,34 +82,34 @@ Graphical Result
|
|||||||
- The shaded part above: Maximum drawdown corresponding to `cum return wo cost`
|
- The shaded part above: Maximum drawdown corresponding to `cum return wo cost`
|
||||||
- The shaded part below: Maximum drawdown corresponding to `cum ex return wo cost`
|
- The shaded part below: Maximum drawdown corresponding to `cum ex return wo cost`
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/report.png
|
.. image:: ../_static/img/analysis/report.png
|
||||||
|
|
||||||
|
|
||||||
Usage of `analysis_position.score_ic`
|
Usage of `analysis_position.score_ic`
|
||||||
-------------------------------------
|
-------------------------------------
|
||||||
|
|
||||||
API
|
API
|
||||||
~~~~~~~~~~~~~~~~
|
~~~
|
||||||
|
|
||||||
.. automodule:: qlib.contrib.report.analysis_position.score_ic
|
.. automodule:: qlib.contrib.report.analysis_position.score_ic
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
Graphical Result
|
Graphical Result
|
||||||
~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
- Axis X: Trading day
|
- Axis X: Trading day
|
||||||
- Axis Y:
|
- Axis Y:
|
||||||
- `ic`
|
- `ic`
|
||||||
The `Pearson correlation coefficient` series between `label` and `prediction score`.
|
The `Pearson correlation coefficient` series between `label` and `prediction score`.
|
||||||
In the above example, the `label` is formulated as `Ref($close, -2)/Ref($close, -1)-1`. Please refer to `Data Feature <data.html#feature>`_ for more details.
|
In the above example, the `label` is formulated as `Ref($close, -2)/Ref($close, -1)-1`. Please refer to `Data Feature <data.html#feature>`_ for more details.
|
||||||
|
|
||||||
- `rank_ic`
|
- `rank_ic`
|
||||||
The `Spearman's rank correlation coefficient` series between `label` and `prediction score`.
|
The `Spearman's rank correlation coefficient` series between `label` and `prediction score`.
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/score_ic.png
|
.. image:: ../_static/img/analysis/score_ic.png
|
||||||
|
|
||||||
|
|
||||||
.. Usage of `analysis_position.cumulative_return`
|
.. Usage of `analysis_position.cumulative_return`
|
||||||
@@ -124,7 +124,7 @@ Graphical Result
|
|||||||
.. Graphical Result
|
.. Graphical Result
|
||||||
.. ~~~~~~~~~~~~~~~~~
|
.. ~~~~~~~~~~~~~~~~~
|
||||||
..
|
..
|
||||||
.. .. note::
|
.. .. note::
|
||||||
..
|
..
|
||||||
.. - Axis X: Trading day
|
.. - Axis X: Trading day
|
||||||
.. - Axis Y:
|
.. - Axis Y:
|
||||||
@@ -134,27 +134,27 @@ Graphical Result
|
|||||||
.. - In the **buy_minus_sell** graph, the **y** value of the **weight** graph at the bottom is `buy_weight + sell_weight`.
|
.. - In the **buy_minus_sell** graph, the **y** value of the **weight** graph at the bottom is `buy_weight + sell_weight`.
|
||||||
.. - In each graph, the **red line** in the histogram on the right represents the average.
|
.. - In each graph, the **red line** in the histogram on the right represents the average.
|
||||||
..
|
..
|
||||||
.. .. image:: ../_static/img/analysis/cumulative_return_buy.png
|
.. .. image:: ../_static/img/analysis/cumulative_return_buy.png
|
||||||
..
|
..
|
||||||
.. .. image:: ../_static/img/analysis/cumulative_return_sell.png
|
.. .. image:: ../_static/img/analysis/cumulative_return_sell.png
|
||||||
..
|
..
|
||||||
.. .. image:: ../_static/img/analysis/cumulative_return_buy_minus_sell.png
|
.. .. image:: ../_static/img/analysis/cumulative_return_buy_minus_sell.png
|
||||||
..
|
..
|
||||||
.. .. image:: ../_static/img/analysis/cumulative_return_hold.png
|
.. .. image:: ../_static/img/analysis/cumulative_return_hold.png
|
||||||
|
|
||||||
|
|
||||||
Usage of `analysis_position.risk_analysis`
|
Usage of `analysis_position.risk_analysis`
|
||||||
----------------------------------------------
|
------------------------------------------
|
||||||
|
|
||||||
API
|
API
|
||||||
~~~~~~~~~~~~~~~~
|
~~~
|
||||||
|
|
||||||
.. automodule:: qlib.contrib.report.analysis_position.risk_analysis
|
.. automodule:: qlib.contrib.report.analysis_position.risk_analysis
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
Graphical Result
|
Graphical Result
|
||||||
~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
@@ -210,7 +210,7 @@ Graphical Result
|
|||||||
The `Standard Deviation` series of monthly `CAR` (cumulative abnormal return) without cost.
|
The `Standard Deviation` series of monthly `CAR` (cumulative abnormal return) without cost.
|
||||||
- `excess_return_with_cost_max_drawdown`
|
- `excess_return_with_cost_max_drawdown`
|
||||||
The `Standard Deviation` series of monthly `CAR` (cumulative abnormal return) with cost.
|
The `Standard Deviation` series of monthly `CAR` (cumulative abnormal return) with cost.
|
||||||
|
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/risk_analysis_annualized_return.png
|
.. image:: ../_static/img/analysis/risk_analysis_annualized_return.png
|
||||||
:align: center
|
:align: center
|
||||||
@@ -221,58 +221,58 @@ Graphical Result
|
|||||||
.. image:: ../_static/img/analysis/risk_analysis_information_ratio.png
|
.. image:: ../_static/img/analysis/risk_analysis_information_ratio.png
|
||||||
:align: center
|
:align: center
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/risk_analysis_std.png
|
.. image:: ../_static/img/analysis/risk_analysis_std.png
|
||||||
:align: center
|
:align: center
|
||||||
|
|
||||||
..
|
..
|
||||||
.. Usage of `analysis_position.rank_label`
|
.. Usage of `analysis_position.rank_label`
|
||||||
.. ----------------------------------------------
|
.. ---------------------------------------
|
||||||
..
|
..
|
||||||
.. API
|
.. API
|
||||||
.. ~~~~~
|
.. ~~~
|
||||||
..
|
..
|
||||||
.. .. automodule:: qlib.contrib.report.analysis_position.rank_label
|
.. .. automodule:: qlib.contrib.report.analysis_position.rank_label
|
||||||
.. :members:
|
.. :members:
|
||||||
..
|
..
|
||||||
..
|
..
|
||||||
.. Graphical Result
|
.. Graphical Result
|
||||||
.. ~~~~~~~~~~~~~~~~~
|
.. ~~~~~~~~~~~~~~~~
|
||||||
..
|
..
|
||||||
.. .. note::
|
.. .. note::
|
||||||
..
|
..
|
||||||
.. - hold/sell/buy graphics:
|
.. - hold/sell/buy graphics:
|
||||||
.. - Axis X: Trading day
|
.. - Axis X: Trading day
|
||||||
.. - Axis Y:
|
.. - Axis Y:
|
||||||
.. Average `ranking ratio`of `label` for stocks that is held/sold/bought on the trading day.
|
.. Average `ranking ratio`of `label` for stocks that is held/sold/bought on the trading day.
|
||||||
..
|
..
|
||||||
.. In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. The `ranking ratio` can be formulated as follows.
|
.. In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. The `ranking ratio` can be formulated as follows.
|
||||||
.. .. math::
|
.. .. math::
|
||||||
..
|
..
|
||||||
.. ranking\ ratio = \frac{Ascending\ Ranking\ of\ label}{Number\ of\ Stocks\ in\ the\ Portfolio}
|
.. ranking\ ratio = \frac{Ascending\ Ranking\ of\ label}{Number\ of\ Stocks\ in\ the\ Portfolio}
|
||||||
..
|
..
|
||||||
.. .. image:: ../_static/img/analysis/rank_label_hold.png
|
.. .. image:: ../_static/img/analysis/rank_label_hold.png
|
||||||
.. :align: center
|
.. :align: center
|
||||||
..
|
..
|
||||||
.. .. image:: ../_static/img/analysis/rank_label_buy.png
|
.. .. image:: ../_static/img/analysis/rank_label_buy.png
|
||||||
.. :align: center
|
.. :align: center
|
||||||
..
|
..
|
||||||
.. .. image:: ../_static/img/analysis/rank_label_sell.png
|
.. .. image:: ../_static/img/analysis/rank_label_sell.png
|
||||||
.. :align: center
|
.. :align: center
|
||||||
..
|
..
|
||||||
..
|
..
|
||||||
|
|
||||||
Usage of `analysis_model.analysis_model_performance`
|
Usage of `analysis_model.analysis_model_performance`
|
||||||
-----------------------------------------------------
|
----------------------------------------------------
|
||||||
|
|
||||||
API
|
API
|
||||||
~~~~~
|
~~~
|
||||||
|
|
||||||
.. automodule:: qlib.contrib.report.analysis_model.analysis_model_performance
|
.. automodule:: qlib.contrib.report.analysis_model.analysis_model_performance
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
Graphical Results
|
Graphical Results
|
||||||
~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
@@ -291,13 +291,13 @@ Graphical Results
|
|||||||
The Difference series between `Cumulative Return` of `Group1` and of `Group5`
|
The Difference series between `Cumulative Return` of `Group1` and of `Group5`
|
||||||
- `long-average`
|
- `long-average`
|
||||||
The Difference series between `Cumulative Return` of `Group1` and average `Cumulative Return` for all stocks.
|
The Difference series between `Cumulative Return` of `Group1` and average `Cumulative Return` for all stocks.
|
||||||
|
|
||||||
The `ranking ratio` can be formulated as follows.
|
The `ranking ratio` can be formulated as follows.
|
||||||
.. math::
|
.. math::
|
||||||
|
|
||||||
ranking\ ratio = \frac{Ascending\ Ranking\ of\ label}{Number\ of\ Stocks\ in\ the\ Portfolio}
|
ranking\ ratio = \frac{Ascending\ Ranking\ of\ label}{Number\ of\ Stocks\ in\ the\ Portfolio}
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/analysis_model_cumulative_return.png
|
.. image:: ../_static/img/analysis/analysis_model_cumulative_return.png
|
||||||
:align: center
|
:align: center
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
@@ -305,7 +305,7 @@ Graphical Results
|
|||||||
The distribution of long-short/long-average returns on each trading day
|
The distribution of long-short/long-average returns on each trading day
|
||||||
|
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/analysis_model_long_short.png
|
.. image:: ../_static/img/analysis/analysis_model_long_short.png
|
||||||
:align: center
|
:align: center
|
||||||
|
|
||||||
.. TODO: ask xiao yang for detial
|
.. TODO: ask xiao yang for detial
|
||||||
@@ -315,14 +315,14 @@ Graphical Results
|
|||||||
- The `Pearson correlation coefficient` series between `labels` and `prediction scores` of stocks in portfolio.
|
- The `Pearson correlation coefficient` series between `labels` and `prediction scores` of stocks in portfolio.
|
||||||
- The graphics reports can be used to evaluate the `prediction scores`.
|
- The graphics reports can be used to evaluate the `prediction scores`.
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/analysis_model_IC.png
|
.. image:: ../_static/img/analysis/analysis_model_IC.png
|
||||||
:align: center
|
:align: center
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
- Monthly IC
|
- Monthly IC
|
||||||
Monthly average of the `Information Coefficient`
|
Monthly average of the `Information Coefficient`
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/analysis_model_monthly_IC.png
|
.. image:: ../_static/img/analysis/analysis_model_monthly_IC.png
|
||||||
:align: center
|
:align: center
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
@@ -331,14 +331,14 @@ Graphical Results
|
|||||||
- IC Normal Dist. Q-Q
|
- IC Normal Dist. Q-Q
|
||||||
The `Quantile-Quantile Plot` is used for the normal distribution of `Information Coefficient` on each trading day.
|
The `Quantile-Quantile Plot` is used for the normal distribution of `Information Coefficient` on each trading day.
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/analysis_model_NDQ.png
|
.. image:: ../_static/img/analysis/analysis_model_NDQ.png
|
||||||
:align: center
|
:align: center
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
- Auto Correlation
|
- Auto Correlation
|
||||||
- The `Pearson correlation coefficient` series between the latest `prediction scores` and the `prediction scores` `lag` days ago of stocks in portfolio on each trading day.
|
- The `Pearson correlation coefficient` series between the latest `prediction scores` and the `prediction scores` `lag` days ago of stocks in portfolio on each trading day.
|
||||||
- The graphics reports can be used to estimate the turnover rate.
|
- The graphics reports can be used to estimate the turnover rate.
|
||||||
|
|
||||||
|
|
||||||
.. image:: ../_static/img/analysis/analysis_model_auto_correlation.png
|
|
||||||
|
.. image:: ../_static/img/analysis/analysis_model_auto_correlation.png
|
||||||
:align: center
|
:align: center
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ Portfolio Strategy: Portfolio Management
|
|||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
|
|
||||||
``Portfolio Strategy`` is designed to adopt different portfolio strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Forecast Model``. Users can use the ``Portfolio Strategy`` in an automatic workflow by ``Workflow`` module, please refer to `Workflow: Workflow Management <workflow.html>`_.
|
``Portfolio Strategy`` is designed to adopt different portfolio strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Forecast Model``. Users can use the ``Portfolio Strategy`` in an automatic workflow by ``Workflow`` module, please refer to `Workflow: Workflow Management <workflow.html>`_.
|
||||||
|
|
||||||
@@ -20,7 +20,7 @@ Base Class & Interface
|
|||||||
======================
|
======================
|
||||||
|
|
||||||
BaseStrategy
|
BaseStrategy
|
||||||
------------------
|
------------
|
||||||
|
|
||||||
Qlib provides a base class ``qlib.strategy.base.BaseStrategy``. All strategy classes need to inherit the base class and implement its interface.
|
Qlib provides a base class ``qlib.strategy.base.BaseStrategy``. All strategy classes need to inherit the base class and implement its interface.
|
||||||
|
|
||||||
@@ -32,7 +32,7 @@ Qlib provides a base class ``qlib.strategy.base.BaseStrategy``. All strategy cla
|
|||||||
Users can inherit `BaseStrategy` to customize their strategy class.
|
Users can inherit `BaseStrategy` to customize their strategy class.
|
||||||
|
|
||||||
WeightStrategyBase
|
WeightStrategyBase
|
||||||
--------------------
|
------------------
|
||||||
|
|
||||||
Qlib also provides a class ``qlib.contrib.strategy.WeightStrategyBase`` that is a subclass of `BaseStrategy`.
|
Qlib also provides a class ``qlib.contrib.strategy.WeightStrategyBase`` that is a subclass of `BaseStrategy`.
|
||||||
|
|
||||||
@@ -60,7 +60,7 @@ Implemented Strategy
|
|||||||
Qlib provides a implemented strategy classes named `TopkDropoutStrategy`.
|
Qlib provides a implemented strategy classes named `TopkDropoutStrategy`.
|
||||||
|
|
||||||
TopkDropoutStrategy
|
TopkDropoutStrategy
|
||||||
------------------
|
-------------------
|
||||||
`TopkDropoutStrategy` is a subclass of `BaseStrategy` and implement the interface `generate_order_list` whose process is as follows.
|
`TopkDropoutStrategy` is a subclass of `BaseStrategy` and implement the interface `generate_order_list` whose process is as follows.
|
||||||
|
|
||||||
- Adopt the ``Topk-Drop`` algorithm to calculate the target amount of each stock
|
- Adopt the ``Topk-Drop`` algorithm to calculate the target amount of each stock
|
||||||
@@ -74,16 +74,16 @@ TopkDropoutStrategy
|
|||||||
In general, the number of stocks currently held is `Topk`, with the exception of being zero at the beginning period of trading.
|
In general, the number of stocks currently held is `Topk`, with the exception of being zero at the beginning period of trading.
|
||||||
For each trading day, let $d$ be the number of the instruments currently held and with a rank $\gt K$ when ranked by the prediction scores from high to low.
|
For each trading day, let $d$ be the number of the instruments currently held and with a rank $\gt K$ when ranked by the prediction scores from high to low.
|
||||||
Then `d` number of stocks currently held with the worst `prediction score` will be sold, and the same number of unheld stocks with the best `prediction score` will be bought.
|
Then `d` number of stocks currently held with the worst `prediction score` will be sold, and the same number of unheld stocks with the best `prediction score` will be bought.
|
||||||
|
|
||||||
In general, $d=$`Drop`, especially when the pool of the candidate instruments is large, $K$ is large, and `Drop` is small.
|
In general, $d=$`Drop`, especially when the pool of the candidate instruments is large, $K$ is large, and `Drop` is small.
|
||||||
|
|
||||||
In most cases, ``TopkDrop`` algorithm sells and buys `Drop` stocks every trading day, which yields a turnover rate of 2$\times$`Drop`/$K$.
|
In most cases, ``TopkDrop`` algorithm sells and buys `Drop` stocks every trading day, which yields a turnover rate of 2$\times$`Drop`/$K$.
|
||||||
|
|
||||||
The following images illustrate a typical scenario.
|
The following images illustrate a typical scenario.
|
||||||
.. image:: ../_static/img/topk_drop.png
|
.. image:: ../_static/img/topk_drop.png
|
||||||
:alt: Topk-Drop
|
:alt: Topk-Drop
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
- Generate the order list from the target amount
|
- Generate the order list from the target amount
|
||||||
|
|
||||||
@@ -98,12 +98,12 @@ and `qlib.contrib.strategy.optimizer.enhanced_indexing.EnhancedIndexingOptimizer
|
|||||||
|
|
||||||
|
|
||||||
Usage & Example
|
Usage & Example
|
||||||
====================
|
===============
|
||||||
|
|
||||||
First, user can create a model to get trading signals(the variable name is ``pred_score`` in following cases).
|
First, user can create a model to get trading signals(the variable name is ``pred_score`` in following cases).
|
||||||
|
|
||||||
Prediction Score
|
Prediction Score
|
||||||
-----------------
|
----------------
|
||||||
|
|
||||||
The `prediction score` is a pandas DataFrame. Its index is <datetime(pd.Timestamp), instrument(str)> and it must
|
The `prediction score` is a pandas DataFrame. Its index is <datetime(pd.Timestamp), instrument(str)> and it must
|
||||||
contains a `score` column.
|
contains a `score` column.
|
||||||
@@ -134,7 +134,7 @@ Qlib didn't add a step to scale the prediction score to a unified scale due to t
|
|||||||
- The model has the flexibility to define the target, loss, and data processing. So we don't think there is a silver bullet to rescale it back directly barely based on the model's outputs. If you want to scale it back to some meaningful values(e.g. stock returns.), an intuitive solution is to create a regression model for the model's recent outputs and your recent target values.
|
- The model has the flexibility to define the target, loss, and data processing. So we don't think there is a silver bullet to rescale it back directly barely based on the model's outputs. If you want to scale it back to some meaningful values(e.g. stock returns.), an intuitive solution is to create a regression model for the model's recent outputs and your recent target values.
|
||||||
|
|
||||||
Running backtest
|
Running backtest
|
||||||
-----------------
|
----------------
|
||||||
|
|
||||||
- In most cases, users could backtest their portfolio management strategy with ``backtest_daily``.
|
- In most cases, users could backtest their portfolio management strategy with ``backtest_daily``.
|
||||||
|
|
||||||
@@ -195,7 +195,7 @@ Running backtest
|
|||||||
|
|
||||||
CSI300_BENCH = "SH000300"
|
CSI300_BENCH = "SH000300"
|
||||||
# Benchmark is for calculating the excess return of your strategy.
|
# Benchmark is for calculating the excess return of your strategy.
|
||||||
# Its data format will be like **ONE normal instrument**.
|
# Its data format will be like **ONE normal instrument**.
|
||||||
# For example, you can query its data with the code below
|
# For example, you can query its data with the code below
|
||||||
# `D.features(["SH000300"], ["$close"], start_time='2010-01-01', end_time='2017-12-31', freq='day')`
|
# `D.features(["SH000300"], ["$close"], start_time='2010-01-01', end_time='2017-12-31', freq='day')`
|
||||||
# It is different from the argument `market`, which indicates a universe of stocks (e.g. **A SET** of stocks like csi300)
|
# It is different from the argument `market`, which indicates a universe of stocks (e.g. **A SET** of stocks like csi300)
|
||||||
@@ -262,7 +262,7 @@ Running backtest
|
|||||||
|
|
||||||
|
|
||||||
Result
|
Result
|
||||||
------------------
|
------
|
||||||
|
|
||||||
The backtest results are in the following form:
|
The backtest results are in the following form:
|
||||||
|
|
||||||
@@ -307,5 +307,5 @@ The backtest results are in the following form:
|
|||||||
|
|
||||||
|
|
||||||
Reference
|
Reference
|
||||||
===================
|
=========
|
||||||
To know more about the `prediction score` `pred_score` output by ``Forecast Model``, please refer to `Forecast Model: Model Training & Prediction <model.html>`_.
|
To know more about the `prediction score` `pred_score` output by ``Forecast Model``, please refer to `Forecast Model: Model Training & Prediction <model.html>`_.
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
.. _workflow:
|
.. _workflow:
|
||||||
|
|
||||||
=================================
|
=============================
|
||||||
Workflow: Workflow Management
|
Workflow: Workflow Management
|
||||||
=================================
|
=============================
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
|
|
||||||
The components in `Qlib Framework <../introduction/introduction.html#framework>`_ are designed in a loosely-coupled way. Users could build their own Quant research workflow with these components like `Example <https://github.com/microsoft/qlib/blob/main/examples/workflow_by_code.py>`_.
|
The components in `Qlib Framework <../introduction/introduction.html#framework>`_ are designed in a loosely-coupled way. Users could build their own Quant research workflow with these components like `Example <https://github.com/microsoft/qlib/blob/main/examples/workflow_by_code.py>`_.
|
||||||
|
|
||||||
@@ -28,7 +28,7 @@ With ``qrun``, user can easily start an `execution`, which includes the followin
|
|||||||
For each `execution`, ``Qlib`` has a complete system to tracking all the information as well as artifacts generated during training, inference and evaluation phase. For more information about how ``Qlib`` handles this, please refer to the related document: `Recorder: Experiment Management <../component/recorder.html>`_.
|
For each `execution`, ``Qlib`` has a complete system to tracking all the information as well as artifacts generated during training, inference and evaluation phase. For more information about how ``Qlib`` handles this, please refer to the related document: `Recorder: Experiment Management <../component/recorder.html>`_.
|
||||||
|
|
||||||
Complete Example
|
Complete Example
|
||||||
===================
|
================
|
||||||
|
|
||||||
Before getting into details, here is a complete example of ``qrun``, which defines the workflow in typical Quant research.
|
Before getting into details, here is a complete example of ``qrun``, which defines the workflow in typical Quant research.
|
||||||
Below is a typical config file of ``qrun``.
|
Below is a typical config file of ``qrun``.
|
||||||
@@ -54,7 +54,7 @@ Below is a typical config file of ``qrun``.
|
|||||||
topk: 50
|
topk: 50
|
||||||
n_drop: 5
|
n_drop: 5
|
||||||
signal:
|
signal:
|
||||||
- <MODEL>
|
- <MODEL>
|
||||||
- <DATASET>
|
- <DATASET>
|
||||||
backtest:
|
backtest:
|
||||||
limit_threshold: 0.095
|
limit_threshold: 0.095
|
||||||
@@ -90,13 +90,13 @@ Below is a typical config file of ``qrun``.
|
|||||||
train: [2008-01-01, 2014-12-31]
|
train: [2008-01-01, 2014-12-31]
|
||||||
valid: [2015-01-01, 2016-12-31]
|
valid: [2015-01-01, 2016-12-31]
|
||||||
test: [2017-01-01, 2020-08-01]
|
test: [2017-01-01, 2020-08-01]
|
||||||
record:
|
record:
|
||||||
- class: SignalRecord
|
- class: SignalRecord
|
||||||
module_path: qlib.workflow.record_temp
|
module_path: qlib.workflow.record_temp
|
||||||
kwargs: {}
|
kwargs: {}
|
||||||
- class: PortAnaRecord
|
- class: PortAnaRecord
|
||||||
module_path: qlib.workflow.record_temp
|
module_path: qlib.workflow.record_temp
|
||||||
kwargs:
|
kwargs:
|
||||||
config: *port_analysis_config
|
config: *port_analysis_config
|
||||||
|
|
||||||
After saving the config into `configuration.yaml`, users could start the workflow and test their ideas with a single command below.
|
After saving the config into `configuration.yaml`, users could start the workflow and test their ideas with a single command below.
|
||||||
@@ -111,22 +111,22 @@ If users want to use ``qrun`` under debug mode, please use the following command
|
|||||||
|
|
||||||
python -m pdb qlib/workflow/cli.py examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml
|
python -m pdb qlib/workflow/cli.py examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
`qrun` will be placed in your $PATH directory when installing ``Qlib``.
|
`qrun` will be placed in your $PATH directory when installing ``Qlib``.
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
The symbol `&` in `yaml` file stands for an anchor of a field, which is useful when another fields include this parameter as part of the value. Taking the configuration file above as an example, users can directly change the value of `market` and `benchmark` without traversing the entire configuration file.
|
The symbol `&` in `yaml` file stands for an anchor of a field, which is useful when another fields include this parameter as part of the value. Taking the configuration file above as an example, users can directly change the value of `market` and `benchmark` without traversing the entire configuration file.
|
||||||
|
|
||||||
|
|
||||||
Configuration File
|
Configuration File
|
||||||
===================
|
==================
|
||||||
|
|
||||||
Let's get into details of ``qrun`` in this section.
|
Let's get into details of ``qrun`` in this section.
|
||||||
Before using ``qrun``, users need to prepare a configuration file. The following content shows how to prepare each part of the configuration file.
|
Before using ``qrun``, users need to prepare a configuration file. The following content shows how to prepare each part of the configuration file.
|
||||||
|
|
||||||
The design logic of the configuration file is very simple. It predefines fixed workflows and provide this yaml interface to users to define how to initialize each component.
|
The design logic of the configuration file is very simple. It predefines fixed workflows and provide this yaml interface to users to define how to initialize each component.
|
||||||
It follow the design of `init_instance_by_config <https://github.com/microsoft/qlib/blob/2aee9e0145decc3e71def70909639b5e5a6f4b58/qlib/utils/__init__.py#L264>`_ . It defines the initialization of each component of Qlib, which typically include the class and the initialization arguments.
|
It follow the design of `init_instance_by_config <https://github.com/microsoft/qlib/blob/2aee9e0145decc3e71def70909639b5e5a6f4b58/qlib/utils/__init__.py#L264>`_ . It defines the initialization of each component of Qlib, which typically include the class and the initialization arguments.
|
||||||
|
|
||||||
For example, the following yaml and code are equivalent.
|
For example, the following yaml and code are equivalent.
|
||||||
@@ -166,7 +166,7 @@ For example, the following yaml and code are equivalent.
|
|||||||
|
|
||||||
|
|
||||||
Qlib Init Section
|
Qlib Init Section
|
||||||
--------------------
|
-----------------
|
||||||
|
|
||||||
At first, the configuration file needs to contain several basic parameters which will be used for qlib initialization.
|
At first, the configuration file needs to contain several basic parameters which will be used for qlib initialization.
|
||||||
|
|
||||||
@@ -181,21 +181,21 @@ The meaning of each field is as follows:
|
|||||||
Type: str. The URI of the Qlib data. For example, it could be the location where the data loaded by ``get_data.py`` are stored.
|
Type: str. The URI of the Qlib data. For example, it could be the location where the data loaded by ``get_data.py`` are stored.
|
||||||
|
|
||||||
- `region`
|
- `region`
|
||||||
- If `region` == "us", ``Qlib`` will be initialized in US-stock mode.
|
- If `region` == "us", ``Qlib`` will be initialized in US-stock mode.
|
||||||
- If `region` == "cn", ``Qlib`` will be initialized in China-stock mode.
|
- If `region` == "cn", ``Qlib`` will be initialized in China-stock mode.
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
The value of `region` should be aligned with the data stored in `provider_uri`.
|
The value of `region` should be aligned with the data stored in `provider_uri`.
|
||||||
|
|
||||||
|
|
||||||
Task Section
|
Task Section
|
||||||
--------------------
|
------------
|
||||||
|
|
||||||
The `task` field in the configuration corresponds to a `task`, which contains the parameters of three different subsections: `Model`, `Dataset` and `Record`.
|
The `task` field in the configuration corresponds to a `task`, which contains the parameters of three different subsections: `Model`, `Dataset` and `Record`.
|
||||||
|
|
||||||
Model Section
|
Model Section
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
In the `task` field, the `model` section describes the parameters of the model to be used for training and inference. For more information about the base ``Model`` class, please refer to `Qlib Model <../component/model.html>`_.
|
In the `task` field, the `model` section describes the parameters of the model to be used for training and inference. For more information about the base ``Model`` class, please refer to `Qlib Model <../component/model.html>`_.
|
||||||
|
|
||||||
@@ -224,14 +224,14 @@ The meaning of each field is as follows:
|
|||||||
Type: str. The path for the model in qlib.
|
Type: str. The path for the model in qlib.
|
||||||
|
|
||||||
- `kwargs`
|
- `kwargs`
|
||||||
The keywords arguments for the model. Please refer to the specific model implementation for more information: `models <https://github.com/microsoft/qlib/blob/main/qlib/contrib/model>`_.
|
The keywords arguments for the model. Please refer to the specific model implementation for more information: `models <https://github.com/microsoft/qlib/blob/main/qlib/contrib/model>`_.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
``Qlib`` provides a util named: ``init_instance_by_config`` to initialize any class inside ``Qlib`` with the configuration includes the fields: `class`, `module_path` and `kwargs`.
|
``Qlib`` provides a util named: ``init_instance_by_config`` to initialize any class inside ``Qlib`` with the configuration includes the fields: `class`, `module_path` and `kwargs`.
|
||||||
|
|
||||||
Dataset Section
|
Dataset Section
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The `dataset` field describes the parameters for the ``Dataset`` module in ``Qlib`` as well those for the module ``DataHandler``. For more information about the ``Dataset`` module, please refer to `Qlib Data <../component/data.html#dataset>`_.
|
The `dataset` field describes the parameters for the ``Dataset`` module in ``Qlib`` as well those for the module ``DataHandler``. For more information about the ``Dataset`` module, please refer to `Qlib Data <../component/data.html#dataset>`_.
|
||||||
|
|
||||||
@@ -266,7 +266,7 @@ Here is the configuration for the ``Dataset`` module which will take care of dat
|
|||||||
test: [2017-01-01, 2020-08-01]
|
test: [2017-01-01, 2020-08-01]
|
||||||
|
|
||||||
Record Section
|
Record Section
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The `record` field is about the parameters the ``Record`` module in ``Qlib``. ``Record`` is responsible for tracking training process and results such as `information Coefficient (IC)` and `backtest` in a standard format.
|
The `record` field is about the parameters the ``Record`` module in ``Qlib``. ``Record`` is responsible for tracking training process and results such as `information Coefficient (IC)` and `backtest` in a standard format.
|
||||||
|
|
||||||
@@ -282,7 +282,7 @@ The following script is the configuration of `backtest` and the `strategy` used
|
|||||||
topk: 50
|
topk: 50
|
||||||
n_drop: 5
|
n_drop: 5
|
||||||
signal:
|
signal:
|
||||||
- <MODEL>
|
- <MODEL>
|
||||||
- <DATASET>
|
- <DATASET>
|
||||||
backtest:
|
backtest:
|
||||||
limit_threshold: 0.095
|
limit_threshold: 0.095
|
||||||
@@ -299,13 +299,13 @@ Here is the configuration details of different `Record Template` such as ``Signa
|
|||||||
|
|
||||||
.. code-block:: YAML
|
.. code-block:: YAML
|
||||||
|
|
||||||
record:
|
record:
|
||||||
- class: SignalRecord
|
- class: SignalRecord
|
||||||
module_path: qlib.workflow.record_temp
|
module_path: qlib.workflow.record_temp
|
||||||
kwargs: {}
|
kwargs: {}
|
||||||
- class: PortAnaRecord
|
- class: PortAnaRecord
|
||||||
module_path: qlib.workflow.record_temp
|
module_path: qlib.workflow.record_temp
|
||||||
kwargs:
|
kwargs:
|
||||||
config: *port_analysis_config
|
config: *port_analysis_config
|
||||||
|
|
||||||
For more information about the ``Record`` module in ``Qlib``, user can refer to the related document: `Record <../component/recorder.html#record-template>`_.
|
For more information about the ``Record`` module in ``Qlib``, user can refer to the related document: `Record <../component/recorder.html#record-template>`_.
|
||||||
|
|||||||
@@ -1,16 +1,16 @@
|
|||||||
.. _code_standard:
|
.. _code_standard:
|
||||||
|
|
||||||
=================================
|
=============
|
||||||
Code Standard
|
Code Standard
|
||||||
=================================
|
=============
|
||||||
|
|
||||||
Docstring
|
Docstring
|
||||||
=================================
|
=========
|
||||||
Please use the `Numpydoc Style <https://stackoverflow.com/a/24385103>`_.
|
Please use the `Numpydoc Style <https://stackoverflow.com/a/24385103>`_.
|
||||||
|
|
||||||
Continuous Integration
|
Continuous Integration
|
||||||
=================================
|
======================
|
||||||
Continuous Integration (CI) tools help you stick to the quality standards by running tests every time you push a new commit and reporting the results to a pull request.
|
Continuous Integration (CI) tools help you stick to the quality standards by running tests every time you push a new commit and reporting the results to a pull request.
|
||||||
|
|
||||||
When you submit a PR request, you can check whether your code passes the CI tests in the "check" section at the bottom of the web page.
|
When you submit a PR request, you can check whether your code passes the CI tests in the "check" section at the bottom of the web page.
|
||||||
|
|
||||||
@@ -23,7 +23,7 @@ When you submit a PR request, you can check whether your code passes the CI test
|
|||||||
python -m black . -l 120
|
python -m black . -l 120
|
||||||
|
|
||||||
|
|
||||||
2. Qlib will check your code style pylint. The checking command is implemented in [github action workflow](https://github.com/microsoft/qlib/blob/0e8b94a552f1c457cfa6cd2c1bb3b87ebb3fb279/.github/workflows/test.yml#L66).
|
2. Qlib will check your code style pylint. The checking command is implemented in [github action workflow](https://github.com/microsoft/qlib/blob/0e8b94a552f1c457cfa6cd2c1bb3b87ebb3fb279/.github/workflows/test.yml#L66).
|
||||||
Sometime pylint's restrictions are not that reasonable. You can ignore specific errors like this
|
Sometime pylint's restrictions are not that reasonable. You can ignore specific errors like this
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
.. _client:
|
.. _client:
|
||||||
|
|
||||||
Qlib Client-Server Framework
|
Qlib Client-Server Framework
|
||||||
===================
|
============================
|
||||||
|
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
-----------
|
------------
|
||||||
Client-Server is designed to solve following problems
|
Client-Server is designed to solve following problems
|
||||||
|
|
||||||
- Manage the data in a centralized way. Users don't have to manage data of different versions.
|
- Manage the data in a centralized way. Users don't have to manage data of different versions.
|
||||||
@@ -159,13 +159,11 @@ Limitations
|
|||||||
2. The rolling operation expression with parameter `0` can not be updated rightly under mechanism of the client-server framework.
|
2. The rolling operation expression with parameter `0` can not be updated rightly under mechanism of the client-server framework.
|
||||||
|
|
||||||
API
|
API
|
||||||
********************
|
***
|
||||||
|
|
||||||
The client is based on `python-socketio<https://python-socketio.readthedocs.io>`_ which is a framework that supports WebSocket client for Python language. The client can only propose requests and receive results, which do not include any calculating procedure.
|
The client is based on `python-socketio<https://python-socketio.readthedocs.io>`_ which is a framework that supports WebSocket client for Python language. The client can only propose requests and receive results, which do not include any calculating procedure.
|
||||||
|
|
||||||
Class
|
Class
|
||||||
--------------------
|
-----
|
||||||
|
|
||||||
.. automodule:: qlib.data.client
|
.. automodule:: qlib.data.client
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
.. _online:
|
.. _online:
|
||||||
|
|
||||||
Online
|
Online
|
||||||
===================
|
======
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
-------------------
|
------------
|
||||||
|
|
||||||
Welcome to use Online, this module simulates what will be like if we do the real trading use our model and strategy.
|
Welcome to use Online, this module simulates what will be like if we do the real trading use our model and strategy.
|
||||||
|
|
||||||
@@ -31,11 +31,11 @@ The file structure can be viewed at fileStruct_.
|
|||||||
|
|
||||||
|
|
||||||
Example
|
Example
|
||||||
-------------------
|
-------
|
||||||
|
|
||||||
Let's take an example,
|
Let's take an example,
|
||||||
|
|
||||||
.. note:: Make sure you have the latest version of `qlib` installed.
|
.. note:: Make sure you have the latest version of `qlib` installed.
|
||||||
|
|
||||||
If you want to use the models and data provided by `qlib`, you only need to do as follows.
|
If you want to use the models and data provided by `qlib`, you only need to do as follows.
|
||||||
|
|
||||||
@@ -93,7 +93,7 @@ If Your account was saved in "./user_data/", you can see the performance of your
|
|||||||
Here 'SH000905' represents csi500 and 'SH000300' represents csi300
|
Here 'SH000905' represents csi500 and 'SH000300' represents csi300
|
||||||
|
|
||||||
Manage your account
|
Manage your account
|
||||||
--------------------
|
-------------------
|
||||||
|
|
||||||
Any account processed by `online` should be saved in a folder. you can use commands
|
Any account processed by `online` should be saved in a folder. you can use commands
|
||||||
defined to manage your accounts.
|
defined to manage your accounts.
|
||||||
@@ -161,7 +161,7 @@ be called at each trading date.
|
|||||||
>> online update -date 2019-10-16 -path ./user_data/
|
>> online update -date 2019-10-16 -path ./user_data/
|
||||||
|
|
||||||
API
|
API
|
||||||
------------------
|
---
|
||||||
|
|
||||||
All those operations are based on defined in `qlib.contrib.online.operator`
|
All those operations are based on defined in `qlib.contrib.online.operator`
|
||||||
|
|
||||||
@@ -170,7 +170,7 @@ All those operations are based on defined in `qlib.contrib.online.operator`
|
|||||||
.. _fileStruct:
|
.. _fileStruct:
|
||||||
|
|
||||||
File structure
|
File structure
|
||||||
------------------
|
--------------
|
||||||
|
|
||||||
'user_data' indicates the root of folder.
|
'user_data' indicates the root of folder.
|
||||||
Name that bold indicates it’s a folder, otherwise it’s a document.
|
Name that bold indicates it’s a folder, otherwise it’s a document.
|
||||||
@@ -214,7 +214,7 @@ Configuration file
|
|||||||
The configure file used in `online` should contain the model and strategy information.
|
The configure file used in `online` should contain the model and strategy information.
|
||||||
|
|
||||||
About the model
|
About the model
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
First, your configuration file needs to have a field about the model,
|
First, your configuration file needs to have a field about the model,
|
||||||
this field and its contents determine the model we used when generating score at predict date.
|
this field and its contents determine the model we used when generating score at predict date.
|
||||||
@@ -243,7 +243,7 @@ contains 2 methods used in `online` module.
|
|||||||
|
|
||||||
|
|
||||||
About the strategy
|
About the strategy
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Your need define the strategy used to generate the order list at predict date.
|
Your need define the strategy used to generate the order list at predict date.
|
||||||
|
|
||||||
@@ -259,7 +259,7 @@ Followings are two examples for a TopkAmountStrategy
|
|||||||
n_drop: 10
|
n_drop: 10
|
||||||
|
|
||||||
Generated files
|
Generated files
|
||||||
------------------
|
---------------
|
||||||
|
|
||||||
The 'online_generate' command will create the order list at {folder_path}/{user_id}/temp/,
|
The 'online_generate' command will create the order list at {folder_path}/{user_id}/temp/,
|
||||||
the name of that is orderlist_{YYYY-MM-DD}.json, YYYY-MM-DD is the date that those orders to be executed.
|
the name of that is orderlist_{YYYY-MM-DD}.json, YYYY-MM-DD is the date that those orders to be executed.
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
.. _tuner:
|
.. _tuner:
|
||||||
|
|
||||||
Tuner
|
Tuner
|
||||||
===================
|
=====
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
-------------------
|
------------
|
||||||
|
|
||||||
Welcome to use Tuner, this document is based on that you can use Estimator proficiently and correctly.
|
Welcome to use Tuner, this document is based on that you can use Estimator proficiently and correctly.
|
||||||
|
|
||||||
@@ -41,19 +41,19 @@ We write a simple configuration example as following,
|
|||||||
tuner_class: QLibTuner
|
tuner_class: QLibTuner
|
||||||
qlib_client:
|
qlib_client:
|
||||||
auto_mount: False
|
auto_mount: False
|
||||||
logging_level: INFO
|
logging_level: INFO
|
||||||
optimization_criteria:
|
optimization_criteria:
|
||||||
report_type: model
|
report_type: model
|
||||||
report_factor: model_score
|
report_factor: model_score
|
||||||
optim_type: max
|
optim_type: max
|
||||||
tuner_pipeline:
|
tuner_pipeline:
|
||||||
-
|
-
|
||||||
model:
|
model:
|
||||||
class: SomeModel
|
class: SomeModel
|
||||||
space: SomeModelSpace
|
space: SomeModelSpace
|
||||||
trainer:
|
trainer:
|
||||||
class: RollingTrainer
|
class: RollingTrainer
|
||||||
strategy:
|
strategy:
|
||||||
class: TopkAmountStrategy
|
class: TopkAmountStrategy
|
||||||
space: TopkAmountStrategySpace
|
space: TopkAmountStrategySpace
|
||||||
max_evals: 2
|
max_evals: 2
|
||||||
@@ -166,13 +166,13 @@ Also, there are some optional fields. The meaning of each field is as follows:
|
|||||||
The class of tuner, str type, must be an already implemented model, such as `QLibTuner` in `qlib`, or a custom tuner, but it must be a subclass of `qlib.contrib.tuner.Tuner`, the default value is `QLibTuner`.
|
The class of tuner, str type, must be an already implemented model, such as `QLibTuner` in `qlib`, or a custom tuner, but it must be a subclass of `qlib.contrib.tuner.Tuner`, the default value is `QLibTuner`.
|
||||||
|
|
||||||
- `tuner_module_path`
|
- `tuner_module_path`
|
||||||
The module path, str type, absolute url is also supported, indicates the path of the implementation of tuner. The default value is `qlib.contrib.tuner.tuner`
|
The module path, str type, absolute url is also supported, indicates the path of the implementation of tuner. The default value is `qlib.contrib.tuner.tuner`
|
||||||
|
|
||||||
About the optimization criteria
|
About the optimization criteria
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
You need to designate a factor to optimize, for tuner need a factor to decide which case is better than other cases.
|
You need to designate a factor to optimize, for tuner need a factor to decide which case is better than other cases.
|
||||||
Usually, we use the result of `estimator`, such as backtest results and the score of model.
|
Usually, we use the result of `estimator`, such as backtest results and the score of model.
|
||||||
|
|
||||||
This part needs contain these fields:
|
This part needs contain these fields:
|
||||||
|
|
||||||
@@ -203,13 +203,13 @@ The tuner pipeline contains different tuners, and the `tuner` program will proce
|
|||||||
.. code-block:: YAML
|
.. code-block:: YAML
|
||||||
|
|
||||||
tuner_pipeline:
|
tuner_pipeline:
|
||||||
-
|
-
|
||||||
model:
|
model:
|
||||||
class: SomeModel
|
class: SomeModel
|
||||||
space: SomeModelSpace
|
space: SomeModelSpace
|
||||||
trainer:
|
trainer:
|
||||||
class: RollingTrainer
|
class: RollingTrainer
|
||||||
strategy:
|
strategy:
|
||||||
class: TopkAmountStrategy
|
class: TopkAmountStrategy
|
||||||
space: TopkAmountStrategySpace
|
space: TopkAmountStrategySpace
|
||||||
max_evals: 2
|
max_evals: 2
|
||||||
@@ -249,25 +249,25 @@ You need to use the same dataset to evaluate your different `estimator` experime
|
|||||||
test_start_date: 2016-07-01
|
test_start_date: 2016-07-01
|
||||||
test_end_date: 2018-04-30
|
test_end_date: 2018-04-30
|
||||||
|
|
||||||
- `rolling_period`
|
- `rolling_period`
|
||||||
The rolling period, integer type, indicates how many time steps need rolling when rolling the data. The default value is `60`. If you use `RollingTrainer`, this config will be used, or it will be ignored.
|
The rolling period, integer type, indicates how many time steps need rolling when rolling the data. The default value is `60`. If you use `RollingTrainer`, this config will be used, or it will be ignored.
|
||||||
|
|
||||||
- `train_start_date`
|
- `train_start_date`
|
||||||
Training start time, str type.
|
Training start time, str type.
|
||||||
|
|
||||||
- `train_end_date`
|
- `train_end_date`
|
||||||
Training end time, str type.
|
Training end time, str type.
|
||||||
|
|
||||||
- `validate_start_date`
|
- `validate_start_date`
|
||||||
Validation start time, str type.
|
Validation start time, str type.
|
||||||
|
|
||||||
- `validate_end_date`
|
- `validate_end_date`
|
||||||
Validation end time, str type.
|
Validation end time, str type.
|
||||||
|
|
||||||
- `test_start_date`
|
- `test_start_date`
|
||||||
Test start time, str type.
|
Test start time, str type.
|
||||||
|
|
||||||
- `test_end_date`
|
- `test_end_date`
|
||||||
Test end time, str type. If `test_end_date` is `-1` or greater than the last date of the data, the last date of the data will be used as `test_end_date`.
|
Test end time, str type. If `test_end_date` is `-1` or greater than the last date of the data, the last date of the data will be used as `test_end_date`.
|
||||||
|
|
||||||
About the data and backtest
|
About the data and backtest
|
||||||
@@ -315,11 +315,10 @@ About the data and backtest
|
|||||||
Experiment Result
|
Experiment Result
|
||||||
-----------------
|
-----------------
|
||||||
|
|
||||||
All the results are stored in experiment file directly, you can check them directly in the corresponding files.
|
All the results are stored in experiment file directly, you can check them directly in the corresponding files.
|
||||||
What we save are as following:
|
What we save are as following:
|
||||||
|
|
||||||
- Global optimal parameters
|
- Global optimal parameters
|
||||||
- Local optimal parameters of each tuner
|
- Local optimal parameters of each tuner
|
||||||
- Config file of this `tuner` experiment
|
- Config file of this `tuner` experiment
|
||||||
- Every `estimator` experiments result in the process
|
- Every `estimator` experiments result in the process
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
============================================================
|
======================
|
||||||
``Qlib`` Documentation
|
``Qlib`` Documentation
|
||||||
============================================================
|
======================
|
||||||
|
|
||||||
``Qlib`` is an AI-oriented quantitative investment platform, which aims to realize the potential, empower the research, and create the value of AI technologies in quantitative investment.
|
``Qlib`` is an AI-oriented quantitative investment platform, which aims to realize the potential, empower the research, and create the value of AI technologies in quantitative investment.
|
||||||
|
|
||||||
@@ -24,12 +24,12 @@ Document Structure
|
|||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 3
|
:maxdepth: 3
|
||||||
:caption: FIRST STEPS:
|
:caption: FIRST STEPS:
|
||||||
|
|
||||||
Installation <start/installation.rst>
|
Installation <start/installation.rst>
|
||||||
Initialization <start/initialization.rst>
|
Initialization <start/initialization.rst>
|
||||||
Data Retrieval <start/getdata.rst>
|
Data Retrieval <start/getdata.rst>
|
||||||
Custom Model Integration <start/integration.rst>
|
Custom Model Integration <start/integration.rst>
|
||||||
|
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 3
|
:maxdepth: 3
|
||||||
@@ -48,7 +48,7 @@ Document Structure
|
|||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 3
|
:maxdepth: 3
|
||||||
:caption: ADVANCED TOPICS:
|
:caption: ADVANCED TOPICS:
|
||||||
|
|
||||||
Building Formulaic Alphas <advanced/alpha.rst>
|
Building Formulaic Alphas <advanced/alpha.rst>
|
||||||
Online & Offline mode <advanced/server.rst>
|
Online & Offline mode <advanced/server.rst>
|
||||||
Serialization <advanced/serial.rst>
|
Serialization <advanced/serial.rst>
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
===============================
|
===============================
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
|
|
||||||
.. image:: ../_static/img/logo/white_bg_rec+word.png
|
.. image:: ../_static/img/logo/white_bg_rec+word.png
|
||||||
:align: center
|
:align: center
|
||||||
@@ -13,8 +13,8 @@ Introduction
|
|||||||
With ``Qlib``, users can easily try their ideas to create better Quant investment strategies.
|
With ``Qlib``, users can easily try their ideas to create better Quant investment strategies.
|
||||||
|
|
||||||
Framework
|
Framework
|
||||||
===================
|
=========
|
||||||
|
|
||||||
.. image:: ../_static/img/framework.svg
|
.. image:: ../_static/img/framework.svg
|
||||||
:align: center
|
:align: center
|
||||||
|
|
||||||
@@ -27,7 +27,7 @@ At the module level, Qlib is a platform that consists of above components. The c
|
|||||||
Name Description
|
Name Description
|
||||||
======================== ==============================================================================
|
======================== ==============================================================================
|
||||||
`Infrastructure` layer `Infrastructure` layer provides underlying support for Quant research.
|
`Infrastructure` layer `Infrastructure` layer provides underlying support for Quant research.
|
||||||
`DataServer` provides high-performance infrastructure for users to manage
|
`DataServer` provides high-performance infrastructure for users to manage
|
||||||
and retrieve raw data. `Trainer` provides flexible interface to control
|
and retrieve raw data. `Trainer` provides flexible interface to control
|
||||||
the training process of models which enable algorithms controlling the
|
the training process of models which enable algorithms controlling the
|
||||||
training process.
|
training process.
|
||||||
@@ -35,13 +35,13 @@ Name Description
|
|||||||
`Workflow` layer `Workflow` layer covers the whole workflow of quantitative investment.
|
`Workflow` layer `Workflow` layer covers the whole workflow of quantitative investment.
|
||||||
`Information Extractor` extracts data for models. `Forecast Model` focuses
|
`Information Extractor` extracts data for models. `Forecast Model` focuses
|
||||||
on producing all kinds of forecast signals (e.g. *alpha*, risk) for other
|
on producing all kinds of forecast signals (e.g. *alpha*, risk) for other
|
||||||
modules. With these signals `Decision Generator` will generate the target
|
modules. With these signals `Decision Generator` will generate the target
|
||||||
trading decisions(i.e. portfolio, orders) to be executed by `Execution Env`
|
trading decisions(i.e. portfolio, orders) to be executed by `Execution Env`
|
||||||
(i.e. the trading market). There may be multiple levels of `Trading Agent`
|
(i.e. the trading market). There may be multiple levels of `Trading Agent`
|
||||||
and `Execution Env` (e.g. an *order executor trading agent and intraday
|
and `Execution Env` (e.g. an *order executor trading agent and intraday
|
||||||
order execution environment* could behave like an interday trading
|
order execution environment* could behave like an interday trading
|
||||||
environment and nested in *daily portfolio management trading agent and
|
environment and nested in *daily portfolio management trading agent and
|
||||||
interday trading environment* )
|
interday trading environment* )
|
||||||
|
|
||||||
`Interface` layer `Interface` layer tries to present a user-friendly interface for the underlying
|
`Interface` layer `Interface` layer tries to present a user-friendly interface for the underlying
|
||||||
system. `Analyser` module will provide users detailed analysis reports of
|
system. `Analyser` module will provide users detailed analysis reports of
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
|
|
||||||
===============================
|
===========
|
||||||
Quick Start
|
Quick Start
|
||||||
===============================
|
===========
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
==============
|
============
|
||||||
|
|
||||||
This ``Quick Start`` guide tries to demonstrate
|
This ``Quick Start`` guide tries to demonstrate
|
||||||
|
|
||||||
@@ -14,7 +14,7 @@ This ``Quick Start`` guide tries to demonstrate
|
|||||||
|
|
||||||
|
|
||||||
Installation
|
Installation
|
||||||
==================
|
============
|
||||||
|
|
||||||
Users can easily intsall ``Qlib`` according to the following steps:
|
Users can easily intsall ``Qlib`` according to the following steps:
|
||||||
|
|
||||||
@@ -34,7 +34,7 @@ Users can easily intsall ``Qlib`` according to the following steps:
|
|||||||
To known more about `installation`, please refer to `Qlib Installation <../start/installation.html>`_.
|
To known more about `installation`, please refer to `Qlib Installation <../start/installation.html>`_.
|
||||||
|
|
||||||
Prepare Data
|
Prepare Data
|
||||||
==============
|
============
|
||||||
|
|
||||||
Load and prepare data by running the following code:
|
Load and prepare data by running the following code:
|
||||||
|
|
||||||
@@ -47,14 +47,14 @@ This dataset is created by public data collected by crawler scripts in ``scripts
|
|||||||
To known more about `prepare data`, please refer to `Data Preparation <../component/data.html#data-preparation>`_.
|
To known more about `prepare data`, please refer to `Data Preparation <../component/data.html#data-preparation>`_.
|
||||||
|
|
||||||
Auto Quant Research Workflow
|
Auto Quant Research Workflow
|
||||||
====================================
|
============================
|
||||||
|
|
||||||
``Qlib`` provides a tool named ``qrun`` to run the whole workflow automatically (including building dataset, training models, backtest and evaluation). Users can start an auto quant research workflow and have a graphical reports analysis according to the following steps:
|
``Qlib`` provides a tool named ``qrun`` to run the whole workflow automatically (including building dataset, training models, backtest and evaluation). Users can start an auto quant research workflow and have a graphical reports analysis according to the following steps:
|
||||||
|
|
||||||
- Quant Research Workflow:
|
- Quant Research Workflow:
|
||||||
- Run ``qrun`` with a config file of the LightGBM model `workflow_config_lightgbm.yaml` as following.
|
- Run ``qrun`` with a config file of the LightGBM model `workflow_config_lightgbm.yaml` as following.
|
||||||
|
|
||||||
.. code-block::
|
.. code-block::
|
||||||
|
|
||||||
cd examples # Avoid running program under the directory contains `qlib`
|
cd examples # Avoid running program under the directory contains `qlib`
|
||||||
qrun benchmarks/LightGBM/workflow_config_lightgbm.yaml
|
qrun benchmarks/LightGBM/workflow_config_lightgbm.yaml
|
||||||
@@ -64,7 +64,7 @@ Auto Quant Research Workflow
|
|||||||
The result of ``qrun`` is as follows, which is also the typical result of ``Forecast model(alpha)``. Please refer to `Intraday Trading <../component/backtest.html>`_. for more details about the result.
|
The result of ``qrun`` is as follows, which is also the typical result of ``Forecast model(alpha)``. Please refer to `Intraday Trading <../component/backtest.html>`_. for more details about the result.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
risk
|
risk
|
||||||
excess_return_without_cost mean 0.000605
|
excess_return_without_cost mean 0.000605
|
||||||
std 0.005481
|
std 0.005481
|
||||||
@@ -77,7 +77,7 @@ Auto Quant Research Workflow
|
|||||||
information_ratio 1.187411
|
information_ratio 1.187411
|
||||||
max_drawdown -0.075024
|
max_drawdown -0.075024
|
||||||
|
|
||||||
|
|
||||||
To know more about `workflow` and `qrun`, please refer to `Workflow: Workflow Management <../component/workflow.html>`_.
|
To know more about `workflow` and `qrun`, please refer to `Workflow: Workflow Management <../component/workflow.html>`_.
|
||||||
|
|
||||||
- Graphical Reports Analysis:
|
- Graphical Reports Analysis:
|
||||||
@@ -89,6 +89,6 @@ Auto Quant Research Workflow
|
|||||||
|
|
||||||
|
|
||||||
Custom Model Integration
|
Custom Model Integration
|
||||||
===============================================
|
========================
|
||||||
|
|
||||||
``Qlib`` provides a batch of models (such as ``lightGBM`` and ``MLP`` models) as examples of ``Forecast Model``. In addition to the default model, users can integrate their own custom models into ``Qlib``. If users are interested in the custom model, please refer to `Custom Model Integration <../start/integration.html>`_.
|
``Qlib`` provides a batch of models (such as ``lightGBM`` and ``MLP`` models) as examples of ``Forecast Model``. In addition to the default model, users can integrate their own custom models into ``Qlib``. If users are interested in the custom model, please refer to `Custom Model Integration <../start/integration.html>`_.
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
.. _api:
|
.. _api:
|
||||||
================================
|
=============
|
||||||
API Reference
|
API Reference
|
||||||
================================
|
=============
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -9,32 +9,32 @@ Here you can find all ``Qlib`` interfaces.
|
|||||||
|
|
||||||
|
|
||||||
Data
|
Data
|
||||||
====================
|
====
|
||||||
|
|
||||||
Provider
|
Provider
|
||||||
--------------------
|
--------
|
||||||
|
|
||||||
.. automodule:: qlib.data.data
|
.. automodule:: qlib.data.data
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Filter
|
Filter
|
||||||
--------------------
|
------
|
||||||
|
|
||||||
.. automodule:: qlib.data.filter
|
.. automodule:: qlib.data.filter
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Class
|
Class
|
||||||
--------------------
|
-----
|
||||||
.. automodule:: qlib.data.base
|
.. automodule:: qlib.data.base
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Operator
|
Operator
|
||||||
--------------------
|
--------
|
||||||
.. automodule:: qlib.data.ops
|
.. automodule:: qlib.data.ops
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Cache
|
Cache
|
||||||
----------------
|
-----
|
||||||
.. autoclass:: qlib.data.cache.MemCacheUnit
|
.. autoclass:: qlib.data.cache.MemCacheUnit
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
@@ -55,7 +55,7 @@ Cache
|
|||||||
|
|
||||||
|
|
||||||
Storage
|
Storage
|
||||||
-------------
|
-------
|
||||||
.. autoclass:: qlib.data.storage.storage.BaseStorage
|
.. autoclass:: qlib.data.storage.storage.BaseStorage
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
@@ -82,52 +82,52 @@ Storage
|
|||||||
|
|
||||||
|
|
||||||
Dataset
|
Dataset
|
||||||
---------------
|
-------
|
||||||
|
|
||||||
Dataset Class
|
Dataset Class
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~
|
||||||
.. automodule:: qlib.data.dataset.__init__
|
.. automodule:: qlib.data.dataset.__init__
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Data Loader
|
Data Loader
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~
|
||||||
.. automodule:: qlib.data.dataset.loader
|
.. automodule:: qlib.data.dataset.loader
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Data Handler
|
Data Handler
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~
|
||||||
.. automodule:: qlib.data.dataset.handler
|
.. automodule:: qlib.data.dataset.handler
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Processor
|
Processor
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~
|
||||||
.. automodule:: qlib.data.dataset.processor
|
.. automodule:: qlib.data.dataset.processor
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
Contrib
|
Contrib
|
||||||
====================
|
=======
|
||||||
|
|
||||||
Model
|
Model
|
||||||
--------------------
|
-----
|
||||||
.. automodule:: qlib.model.base
|
.. automodule:: qlib.model.base
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Strategy
|
Strategy
|
||||||
-------------------
|
--------
|
||||||
|
|
||||||
.. automodule:: qlib.contrib.strategy.strategy
|
.. automodule:: qlib.contrib.strategy.strategy
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Evaluate
|
Evaluate
|
||||||
-----------------
|
--------
|
||||||
|
|
||||||
.. automodule:: qlib.contrib.evaluate
|
.. automodule:: qlib.contrib.evaluate
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
Report
|
Report
|
||||||
-----------------
|
------
|
||||||
|
|
||||||
.. automodule:: qlib.contrib.report.analysis_position.report
|
.. automodule:: qlib.contrib.report.analysis_position.report
|
||||||
:members:
|
:members:
|
||||||
@@ -159,103 +159,100 @@ Report
|
|||||||
|
|
||||||
|
|
||||||
Workflow
|
Workflow
|
||||||
====================
|
========
|
||||||
|
|
||||||
|
|
||||||
Experiment Manager
|
Experiment Manager
|
||||||
--------------------
|
------------------
|
||||||
.. autoclass:: qlib.workflow.expm.ExpManager
|
.. autoclass:: qlib.workflow.expm.ExpManager
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Experiment
|
Experiment
|
||||||
--------------------
|
----------
|
||||||
.. autoclass:: qlib.workflow.exp.Experiment
|
.. autoclass:: qlib.workflow.exp.Experiment
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Recorder
|
Recorder
|
||||||
--------------------
|
--------
|
||||||
.. autoclass:: qlib.workflow.recorder.Recorder
|
.. autoclass:: qlib.workflow.recorder.Recorder
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Record Template
|
Record Template
|
||||||
--------------------
|
---------------
|
||||||
.. automodule:: qlib.workflow.record_temp
|
.. automodule:: qlib.workflow.record_temp
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Task Management
|
Task Management
|
||||||
====================
|
===============
|
||||||
|
|
||||||
|
|
||||||
TaskGen
|
TaskGen
|
||||||
--------------------
|
-------
|
||||||
.. automodule:: qlib.workflow.task.gen
|
.. automodule:: qlib.workflow.task.gen
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
TaskManager
|
TaskManager
|
||||||
--------------------
|
-----------
|
||||||
.. automodule:: qlib.workflow.task.manage
|
.. automodule:: qlib.workflow.task.manage
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Trainer
|
Trainer
|
||||||
--------------------
|
-------
|
||||||
.. automodule:: qlib.model.trainer
|
.. automodule:: qlib.model.trainer
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Collector
|
Collector
|
||||||
--------------------
|
---------
|
||||||
.. automodule:: qlib.workflow.task.collect
|
.. automodule:: qlib.workflow.task.collect
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Group
|
Group
|
||||||
--------------------
|
-----
|
||||||
.. automodule:: qlib.model.ens.group
|
.. automodule:: qlib.model.ens.group
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Ensemble
|
Ensemble
|
||||||
--------------------
|
--------
|
||||||
.. automodule:: qlib.model.ens.ensemble
|
.. automodule:: qlib.model.ens.ensemble
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Utils
|
Utils
|
||||||
--------------------
|
-----
|
||||||
.. automodule:: qlib.workflow.task.utils
|
.. automodule:: qlib.workflow.task.utils
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
Online Serving
|
Online Serving
|
||||||
====================
|
==============
|
||||||
|
|
||||||
|
|
||||||
Online Manager
|
Online Manager
|
||||||
--------------------
|
--------------
|
||||||
.. automodule:: qlib.workflow.online.manager
|
.. automodule:: qlib.workflow.online.manager
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Online Strategy
|
Online Strategy
|
||||||
--------------------
|
---------------
|
||||||
.. automodule:: qlib.workflow.online.strategy
|
.. automodule:: qlib.workflow.online.strategy
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
Online Tool
|
Online Tool
|
||||||
--------------------
|
-----------
|
||||||
.. automodule:: qlib.workflow.online.utils
|
.. automodule:: qlib.workflow.online.utils
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
RecordUpdater
|
RecordUpdater
|
||||||
--------------------
|
-------------
|
||||||
.. automodule:: qlib.workflow.online.update
|
.. automodule:: qlib.workflow.online.update
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
Utils
|
Utils
|
||||||
====================
|
=====
|
||||||
|
|
||||||
Serializable
|
Serializable
|
||||||
--------------------
|
------------
|
||||||
|
|
||||||
.. automodule:: qlib.utils.serial.Serializable
|
.. automodule:: qlib.utils.serial.Serializable
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,18 +1,18 @@
|
|||||||
.. _getdata:
|
.. _getdata:
|
||||||
|
|
||||||
=============================
|
==============
|
||||||
Data Retrieval
|
Data Retrieval
|
||||||
=============================
|
==============
|
||||||
|
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
====================
|
============
|
||||||
|
|
||||||
Users can get stock data with ``Qlib``. The following examples demonstrate the basic user interface.
|
Users can get stock data with ``Qlib``. The following examples demonstrate the basic user interface.
|
||||||
|
|
||||||
Examples
|
Examples
|
||||||
====================
|
========
|
||||||
|
|
||||||
|
|
||||||
``QLib`` Initialization:
|
``QLib`` Initialization:
|
||||||
@@ -30,7 +30,7 @@ If users followed steps in `initialization <initialization.html>`_ and downloade
|
|||||||
Load trading calendar with given time range and frequency:
|
Load trading calendar with given time range and frequency:
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
>> from qlib.data import D
|
>> from qlib.data import D
|
||||||
>> D.calendar(start_time='2010-01-01', end_time='2017-12-31', freq='day')[:2]
|
>> D.calendar(start_time='2010-01-01', end_time='2017-12-31', freq='day')[:2]
|
||||||
[Timestamp('2010-01-04 00:00:00'), Timestamp('2010-01-05 00:00:00')]
|
[Timestamp('2010-01-04 00:00:00'), Timestamp('2010-01-05 00:00:00')]
|
||||||
@@ -46,7 +46,7 @@ Parse a given market name into a stock pool config:
|
|||||||
Load instruments of certain stock pool in the given time range:
|
Load instruments of certain stock pool in the given time range:
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
>> from qlib.data import D
|
>> from qlib.data import D
|
||||||
>> instruments = D.instruments(market='csi300')
|
>> instruments = D.instruments(market='csi300')
|
||||||
>> D.list_instruments(instruments=instruments, start_time='2010-01-01', end_time='2017-12-31', as_list=True)[:6]
|
>> D.list_instruments(instruments=instruments, start_time='2010-01-01', end_time='2017-12-31', as_list=True)[:6]
|
||||||
@@ -79,14 +79,14 @@ For more details about filter, please refer `Filter API <../component/data.html>
|
|||||||
Load features of certain instruments in a given time range:
|
Load features of certain instruments in a given time range:
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
>> from qlib.data import D
|
>> from qlib.data import D
|
||||||
>> instruments = ['SH600000']
|
>> instruments = ['SH600000']
|
||||||
>> fields = ['$close', '$volume', 'Ref($close, 1)', 'Mean($close, 3)', '$high-$low']
|
>> fields = ['$close', '$volume', 'Ref($close, 1)', 'Mean($close, 3)', '$high-$low']
|
||||||
>> D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31', freq='day').head()
|
>> D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31', freq='day').head()
|
||||||
|
|
||||||
$close $volume Ref($close, 1) Mean($close, 3) $high-$low
|
$close $volume Ref($close, 1) Mean($close, 3) $high-$low
|
||||||
instrument datetime
|
instrument datetime
|
||||||
SH600000 2010-01-04 86.778313 16162960.0 88.825928 88.061483 2.907631
|
SH600000 2010-01-04 86.778313 16162960.0 88.825928 88.061483 2.907631
|
||||||
2010-01-05 87.433578 28117442.0 86.778313 87.679273 3.235252
|
2010-01-05 87.433578 28117442.0 86.778313 87.679273 3.235252
|
||||||
2010-01-06 85.713585 23632884.0 87.433578 86.641825 1.720009
|
2010-01-06 85.713585 23632884.0 87.433578 86.641825 1.720009
|
||||||
@@ -108,7 +108,7 @@ Load features of certain stock pool in a given time range:
|
|||||||
>> D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31', freq='day').head()
|
>> D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31', freq='day').head()
|
||||||
|
|
||||||
$close $volume Ref($close, 1) Mean($close, 3) $high-$low
|
$close $volume Ref($close, 1) Mean($close, 3) $high-$low
|
||||||
instrument datetime
|
instrument datetime
|
||||||
SH600655 2010-01-04 2699.567383 158193.328125 2619.070312 2626.097738 124.580566
|
SH600655 2010-01-04 2699.567383 158193.328125 2619.070312 2626.097738 124.580566
|
||||||
2010-01-08 2612.359619 77501.406250 2584.567627 2623.220133 83.373047
|
2010-01-08 2612.359619 77501.406250 2584.567627 2623.220133 83.373047
|
||||||
2010-01-11 2712.982422 160852.390625 2612.359619 2636.636556 146.621582
|
2010-01-11 2712.982422 160852.390625 2612.359619 2636.636556 146.621582
|
||||||
@@ -147,5 +147,5 @@ Here is an exmaple which does the same thing as above examples.
|
|||||||
|
|
||||||
|
|
||||||
API
|
API
|
||||||
====================
|
===
|
||||||
To know more about how to use the Data, go to API Reference: `Data API <../reference/api.html#data>`_
|
To know more about how to use the Data, go to API Reference: `Data API <../reference/api.html#data>`_
|
||||||
|
|||||||
@@ -1,23 +1,23 @@
|
|||||||
.. _initialization:
|
.. _initialization:
|
||||||
|
|
||||||
====================
|
===================
|
||||||
Qlib Initialization
|
Qlib Initialization
|
||||||
====================
|
===================
|
||||||
|
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
|
|
||||||
Initialization
|
Initialization
|
||||||
=========================
|
==============
|
||||||
|
|
||||||
Please follow the steps below to initialize ``Qlib``.
|
Please follow the steps below to initialize ``Qlib``.
|
||||||
|
|
||||||
Download and prepare the Data: execute the following command to download stock data. Please pay `attention` that the data is collected from `Yahoo Finance <https://finance.yahoo.com/lookup>`_ and the data might not be perfect. We recommend users to prepare their own data if they have high-quality datasets. Please refer to `Data <../component/data.html#converting-csv-format-into-qlib-format>`_ for more information about customized dataset.
|
Download and prepare the Data: execute the following command to download stock data. Please pay `attention` that the data is collected from `Yahoo Finance <https://finance.yahoo.com/lookup>`_ and the data might not be perfect. We recommend users to prepare their own data if they have high-quality datasets. Please refer to `Data <../component/data.html#converting-csv-format-into-qlib-format>`_ for more information about customized dataset.
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
python scripts/get_data.py qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn
|
python scripts/get_data.py qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn
|
||||||
|
|
||||||
Please refer to `Data Preparation <../component/data.html#data-preparation>`_ for more information about `get_data.py`,
|
Please refer to `Data Preparation <../component/data.html#data-preparation>`_ for more information about `get_data.py`,
|
||||||
|
|
||||||
|
|
||||||
@@ -30,7 +30,7 @@ Initialize Qlib before calling other APIs: run following code in python.
|
|||||||
from qlib.constant import REG_CN
|
from qlib.constant import REG_CN
|
||||||
provider_uri = "~/.qlib/qlib_data/cn_data" # target_dir
|
provider_uri = "~/.qlib/qlib_data/cn_data" # target_dir
|
||||||
qlib.init(provider_uri=provider_uri, region=REG_CN)
|
qlib.init(provider_uri=provider_uri, region=REG_CN)
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
Do not import qlib package in the repository directory of ``Qlib``, otherwise, errors may occur.
|
Do not import qlib package in the repository directory of ``Qlib``, otherwise, errors may occur.
|
||||||
|
|
||||||
@@ -56,16 +56,16 @@ The following are several important parameters of `qlib.init` (`Qlib` has a lot
|
|||||||
- `redis_port`
|
- `redis_port`
|
||||||
Type: int, optional parameter(default: 6379), port of `redis`
|
Type: int, optional parameter(default: 6379), port of `redis`
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
The value of `region` should be aligned with the data stored in `provider_uri`. Currently, ``scripts/get_data.py`` only provides China stock market data. If users want to use the US stock market data, they should prepare their own US-stock data in `provider_uri` and switch to US-stock mode.
|
The value of `region` should be aligned with the data stored in `provider_uri`. Currently, ``scripts/get_data.py`` only provides China stock market data. If users want to use the US stock market data, they should prepare their own US-stock data in `provider_uri` and switch to US-stock mode.
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
If Qlib fails to connect redis via `redis_host` and `redis_port`, cache mechanism will not be used! Please refer to `Cache <../component/data.html#cache>`_ for details.
|
If Qlib fails to connect redis via `redis_host` and `redis_port`, cache mechanism will not be used! Please refer to `Cache <../component/data.html#cache>`_ for details.
|
||||||
- `exp_manager`
|
- `exp_manager`
|
||||||
Type: dict, optional parameter, the setting of `experiment manager` to be used in qlib. Users can specify an experiment manager class, as well as the tracking URI for all the experiments. However, please be aware that we only support input of a dictionary in the following style for `exp_manager`. For more information about `exp_manager`, users can refer to `Recorder: Experiment Management <../component/recorder.html>`_.
|
Type: dict, optional parameter, the setting of `experiment manager` to be used in qlib. Users can specify an experiment manager class, as well as the tracking URI for all the experiments. However, please be aware that we only support input of a dictionary in the following style for `exp_manager`. For more information about `exp_manager`, users can refer to `Recorder: Experiment Management <../component/recorder.html>`_.
|
||||||
|
|
||||||
.. code-block:: Python
|
.. code-block:: Python
|
||||||
|
|
||||||
# For example, if you want to set your tracking_uri to a <specific folder>, you can initialize qlib below
|
# For example, if you want to set your tracking_uri to a <specific folder>, you can initialize qlib below
|
||||||
@@ -78,7 +78,7 @@ The following are several important parameters of `qlib.init` (`Qlib` has a lot
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
- `mongo`
|
- `mongo`
|
||||||
Type: dict, optional parameter, the setting of `MongoDB <https://www.mongodb.com/>`_ which will be used in some features such as `Task Management <../advanced/task_management.html>`_, with high performance and clustered processing.
|
Type: dict, optional parameter, the setting of `MongoDB <https://www.mongodb.com/>`_ which will be used in some features such as `Task Management <../advanced/task_management.html>`_, with high performance and clustered processing.
|
||||||
Users need to follow the steps in `installation <https://www.mongodb.com/try/download/community>`_ to install MongoDB firstly and then access it via a URI.
|
Users need to follow the steps in `installation <https://www.mongodb.com/try/download/community>`_ to install MongoDB firstly and then access it via a URI.
|
||||||
Users can access mongodb with credential by setting "task_url" to a string like `"mongodb://%s:%s@%s" % (user, pwd, host + ":" + port)`.
|
Users can access mongodb with credential by setting "task_url" to a string like `"mongodb://%s:%s@%s" % (user, pwd, host + ":" + port)`.
|
||||||
|
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
.. _installation:
|
.. _installation:
|
||||||
|
|
||||||
====================
|
============
|
||||||
Installation
|
Installation
|
||||||
====================
|
============
|
||||||
|
|
||||||
.. currentmodule:: qlib
|
.. currentmodule:: qlib
|
||||||
|
|
||||||
@@ -24,7 +24,7 @@ Also, Users can install ``Qlib`` by the source code according to the following s
|
|||||||
|
|
||||||
- Enter the root directory of ``Qlib``, in which the file ``setup.py`` exists.
|
- Enter the root directory of ``Qlib``, in which the file ``setup.py`` exists.
|
||||||
- Then, please execute the following command to install the environment dependencies and install ``Qlib``:
|
- Then, please execute the following command to install the environment dependencies and install ``Qlib``:
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
$ pip install numpy
|
$ pip install numpy
|
||||||
@@ -34,7 +34,7 @@ Also, Users can install ``Qlib`` by the source code according to the following s
|
|||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
It's recommended to use anaconda/miniconda to setup the environment. ``Qlib`` needs lightgbm and pytorch packages, use pip to install them.
|
It's recommended to use anaconda/miniconda to setup the environment. ``Qlib`` needs lightgbm and pytorch packages, use pip to install them.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Use the following code to make sure the installation successful:
|
Use the following code to make sure the installation successful:
|
||||||
@@ -44,6 +44,3 @@ Use the following code to make sure the installation successful:
|
|||||||
>>> import qlib
|
>>> import qlib
|
||||||
>>> qlib.__version__
|
>>> qlib.__version__
|
||||||
<LATEST VERSION>
|
<LATEST VERSION>
|
||||||
|
|
||||||
|
|
||||||
=====================
|
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
=========================================
|
========================
|
||||||
Custom Model Integration
|
Custom Model Integration
|
||||||
=========================================
|
========================
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
===================
|
============
|
||||||
|
|
||||||
``Qlib``'s `Model Zoo` includes models such as ``LightGBM``, ``MLP``, ``LSTM``, etc.. These models are examples of ``Forecast Model``. In addition to the default models ``Qlib`` provide, users can integrate their own custom models into ``Qlib``.
|
``Qlib``'s `Model Zoo` includes models such as ``LightGBM``, ``MLP``, ``LSTM``, etc.. These models are examples of ``Forecast Model``. In addition to the default models ``Qlib`` provide, users can integrate their own custom models into ``Qlib``.
|
||||||
|
|
||||||
@@ -14,7 +14,7 @@ Users can integrate their own custom models according to the following steps.
|
|||||||
- Test the custom model.
|
- Test the custom model.
|
||||||
|
|
||||||
Custom Model Class
|
Custom Model Class
|
||||||
===========================
|
==================
|
||||||
The Custom models need to inherit `qlib.model.base.Model <../reference/api.html#module-qlib.model.base>`_ and override the methods in it.
|
The Custom models need to inherit `qlib.model.base.Model <../reference/api.html#module-qlib.model.base>`_ and override the methods in it.
|
||||||
|
|
||||||
- Override the `__init__` method
|
- Override the `__init__` method
|
||||||
@@ -36,7 +36,7 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html#
|
|||||||
- The parameters could include some `optional` parameters with default values, such as `num_boost_round = 1000` for `GBDT`.
|
- The parameters could include some `optional` parameters with default values, such as `num_boost_round = 1000` for `GBDT`.
|
||||||
- Code Example: In the following example, `num_boost_round = 1000` is an optional parameter.
|
- Code Example: In the following example, `num_boost_round = 1000` is an optional parameter.
|
||||||
.. code-block:: Python
|
.. code-block:: Python
|
||||||
|
|
||||||
def fit(self, dataset: DatasetH, num_boost_round = 1000, **kwargs):
|
def fit(self, dataset: DatasetH, num_boost_round = 1000, **kwargs):
|
||||||
|
|
||||||
# prepare dataset for lgb training and evaluation
|
# prepare dataset for lgb training and evaluation
|
||||||
@@ -101,14 +101,14 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html#
|
|||||||
)
|
)
|
||||||
|
|
||||||
Configuration File
|
Configuration File
|
||||||
=======================
|
==================
|
||||||
|
|
||||||
The configuration file is described in detail in the `Workflow <../component/workflow.html#complete-example>`_ document. In order to integrate the custom model into ``Qlib``, users need to modify the "model" field in the configuration file. The configuration describes which models to use and how we can initialize it.
|
The configuration file is described in detail in the `Workflow <../component/workflow.html#complete-example>`_ document. In order to integrate the custom model into ``Qlib``, users need to modify the "model" field in the configuration file. The configuration describes which models to use and how we can initialize it.
|
||||||
|
|
||||||
- Example: The following example describes the `model` field of configuration file about the custom lightgbm model mentioned above, where `module_path` is the module path, `class` is the class name, and `args` is the hyperparameter passed into the __init__ method. All parameters in the field is passed to `self._params` by `\*\*kwargs` in `__init__` except `loss = mse`.
|
- Example: The following example describes the `model` field of configuration file about the custom lightgbm model mentioned above, where `module_path` is the module path, `class` is the class name, and `args` is the hyperparameter passed into the __init__ method. All parameters in the field is passed to `self._params` by `\*\*kwargs` in `__init__` except `loss = mse`.
|
||||||
|
|
||||||
.. code-block:: YAML
|
.. code-block:: YAML
|
||||||
|
|
||||||
model:
|
model:
|
||||||
class: LGBModel
|
class: LGBModel
|
||||||
module_path: qlib.contrib.model.gbdt
|
module_path: qlib.contrib.model.gbdt
|
||||||
@@ -126,7 +126,7 @@ The configuration file is described in detail in the `Workflow <../component/wor
|
|||||||
Users could find configuration file of the baselines of the ``Model`` in ``examples/benchmarks``. All the configurations of different models are listed under the corresponding model folder.
|
Users could find configuration file of the baselines of the ``Model`` in ``examples/benchmarks``. All the configurations of different models are listed under the corresponding model folder.
|
||||||
|
|
||||||
Model Testing
|
Model Testing
|
||||||
=====================
|
=============
|
||||||
Assuming that the configuration file is ``examples/benchmarks/LightGBM/workflow_config_lightgbm.yaml``, users can run the following command to test the custom model:
|
Assuming that the configuration file is ``examples/benchmarks/LightGBM/workflow_config_lightgbm.yaml``, users can run the following command to test the custom model:
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
@@ -136,10 +136,10 @@ Assuming that the configuration file is ``examples/benchmarks/LightGBM/workflow_
|
|||||||
|
|
||||||
.. note:: ``qrun`` is a built-in command of ``Qlib``.
|
.. note:: ``qrun`` is a built-in command of ``Qlib``.
|
||||||
|
|
||||||
Also, ``Model`` can also be tested as a single module. An example has been given in ``examples/workflow_by_code.ipynb``.
|
Also, ``Model`` can also be tested as a single module. An example has been given in ``examples/workflow_by_code.ipynb``.
|
||||||
|
|
||||||
|
|
||||||
Reference
|
Reference
|
||||||
=====================
|
=========
|
||||||
|
|
||||||
To know more about ``Forecast Model``, please refer to `Forecast Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.model.base>`_.
|
To know more about ``Forecast Model``, please refer to `Forecast Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.model.base>`_.
|
||||||
|
|||||||
Reference in New Issue
Block a user