diff --git a/README.md b/README.md index cee88922f..75631d95e 100644 --- a/README.md +++ b/README.md @@ -11,14 +11,14 @@ With Qlib, you can easily try your ideas to create better Quant investment strat For more details, please refer to our paper ["Qlib: An AI-oriented Quantitative Investment Platform"](https://arxiv.org/abs/2009.11189). -- [Framework of Qlib](#framework-of-qlib) +- [Framework of Qlib](#Framework-of-Qlib) - [Quick Start](#Quick-Start) - [Installation](#Installation) - [Data Preparation](#Data-Preparation) - [Auto Quant Research Workflow with](#Auto-Quant-Research-Workflow) - [Building Customized Quant Research Workflow by Code](#Building-Customized-Quant-Research-Workflow-by-Code) - [More About Qlib](#More-About-Qlib) -- [Offline mode and online mode of data server](#Offline-Mode-and-Online-Mode-of-the-Data-Server) +- [Offline Mode and Online Mode](#Offline-Mode-and-Online-Mode) - [Performance of Qlib Data Server](#Performance-of-Qlib-Data-Server) - [Contributing](#Contributing) @@ -183,12 +183,12 @@ Qlib is in active and continuing development. Our plan is in the roadmap, which -# Offline Mode and Online Mode of the Data Server -The data server of Qlib can either deployed as offline mode or online mode. The default mode is offline mode. +# Offline Mode and Online Mode +The data server of Qlib can either deployed as `Offline` mode or `Online` mode. The default mode is offline mode. -Under offline mode, the data will be deployed locally. +Under `Offline` mode, the data will be deployed locally. -Under online mode, the data will be deployed as a shared data service. The data and their cache will be shared by all the clients. The data retrieval performance is expected to be improved due to a higher rate of cache hits. It will consume less disk space, too. The documents of the online mode can be found in [Qlib-Server](https://qlib-server.readthedocs.io/). The online mode can be deployed automatically with [Azure CLI based scripts](https://qlib-server.readthedocs.io/en/latest/build.html#one-click-deployment-in-azure). The source code of online data server can be found in [qlib-server repository](https://github.com/microsoft/qlib-server). +Under `Online` mode, the data will be deployed as a shared data service. The data and their cache will be shared by all the clients. The data retrieval performance is expected to be improved due to a higher rate of cache hits. It will consume less disk space, too. The documents of the online mode can be found in [Qlib-Server](https://qlib-server.readthedocs.io/). The online mode can be deployed automatically with [Azure CLI based scripts](https://qlib-server.readthedocs.io/en/latest/build.html#one-click-deployment-in-azure). The source code of online data server can be found in [Qlib-Server repository](https://github.com/microsoft/qlib-server). ## Performance of Qlib Data Server The performance of data processing is important to data-driven methods like AI technologies. As an AI-oriented platform, Qlib provides a solution for data storage and data processing. To demonstrate the performance of Qlib data server, we diff --git a/docs/advanced/alpha.rst b/docs/advanced/alpha.rst index ba58b924f..08477d4b6 100644 --- a/docs/advanced/alpha.rst +++ b/docs/advanced/alpha.rst @@ -49,52 +49,53 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib: .. code-block:: python - >>> from qlib.contrib.estimator.handler import QLibDataHandler - >>> fields = ['(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close'] # MACD - >>> names = ['MACD'] - >>> labels = ['Ref($vwap, -2)/Ref($vwap, -1) - 1'] # label - >>> label_names = ['LABEL'] - >>> data_handler = QLibDataHandler(start_date='2010-01-01', end_date='2017-12-31', fields=fields, names=names, labels=labels, label_names=label_names) - >>> TRAINER_CONFIG = { - ... "train_start_date": "2007-01-01", - ... "train_end_date": "2014-12-31", - ... "validate_start_date": "2015-01-01", - ... "validate_end_date": "2016-12-31", - ... "test_start_date": "2017-01-01", - ... "test_end_date": "2020-08-01", - ... } - >>> feature_train, label_train, feature_validate, label_validate, feature_test, label_test = data_handler.get_split_data(**TRAINER_CONFIG) - >>> print(feature_train, label_train) - MACD + >> from qlib.contrib.estimator.handler import QLibDataHandler + >> MACD_EXP = '(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close' + >> fields = [MACD_EXP] # MACD + >> names = ['MACD'] + >> labels = ['$close'] # label + >> label_names = ['LABEL'] + >> data_handler = QLibDataHandler(start_date='2010-01-01', end_date='2017-12-31', fields=fields, names=names, labels=labels, label_names=label_names) + >> TRAINER_CONFIG = { + .. "train_start_date": "2007-01-01", + .. "train_end_date": "2014-12-31", + .. "validate_start_date": "2015-01-01", + .. "validate_end_date": "2016-12-31", + .. "test_start_date": "2017-01-01", + .. "test_end_date": "2020-08-01", + .. } + >> feature_train, label_train, feature_validate, label_validate, feature_test, label_test = data_handler.get_split_data(**TRAINER_CONFIG) + >> print(feature_train, label_train) + MACD instrument datetime - SH600004 2012-01-04 -0.030853 - 2012-01-05 -0.030452 - 2012-01-06 -0.028252 - 2012-01-09 -0.024507 - 2012-01-10 -0.019744 + SH600000 2010-01-04 -0.008625 + 2010-01-05 -0.007234 + 2010-01-06 -0.007693 + 2010-01-07 -0.009633 + 2010-01-08 -0.009891 ... ... - SZ300273 2014-12-25 0.031339 - 2014-12-26 0.029695 - 2014-12-29 0.025577 - 2014-12-30 0.020493 - 2014-12-31 0.017089 + SZ300251 2014-12-25 0.043072 + 2014-12-26 0.041345 + 2014-12-29 0.042733 + 2014-12-30 0.042066 + 2014-12-31 0.036299 - [605882 rows x 1 columns] - label + [322025 rows x 1 columns] + LABEL instrument datetime - SH600004 2012-01-04 0.003021 - 2012-01-05 0.017434 - 2012-01-06 0.015490 - 2012-01-09 0.002324 - 2012-01-10 -0.002542 + SH600000 2010-01-04 4.260015 + 2010-01-05 4.292182 + 2010-01-06 4.207747 + 2010-01-07 4.113258 + 2010-01-08 4.159496 ... ... - SZ300273 2014-12-25 -0.032454 - 2014-12-26 -0.016638 - 2014-12-29 0.008263 - 2014-12-30 -0.011985 - 2014-12-31 0.047797 + SZ300251 2014-12-25 4.343212 + 2014-12-26 4.470587 + 2014-12-29 4.762474 + 2014-12-30 4.369748 + 2014-12-31 4.182222 - [605882 rows x 1 columns] + [322025 rows x 1 columns] Reference =========== diff --git a/docs/component/backtest.rst b/docs/component/backtest.rst index 2d9f3a25b..fd4ac19fa 100644 --- a/docs/component/backtest.rst +++ b/docs/component/backtest.rst @@ -31,9 +31,9 @@ The simple example of the default strategy is as follows. # pred_score is the prediction score report, positions = backtest(pred_score, topk=50, n_drop=0.5, verbose=False, limit_threshold=0.0095) -To know more about backtesting with a specific strategy, please refer to `Strategy `_. +To know more about backtesting with a specific ``Strategy``, please refer to `Strategy `_. -To know more about the prediction score `pred_score` output by ``Model``, please refer to `Interday Model: Model Training & Prediction `_. +To know more about the prediction score `pred_score` output by ``Interday Model``, please refer to `Interday Model: Model Training & Prediction `_. Prediction Score ----------------- @@ -58,7 +58,7 @@ A prediction sample is shown as follows. SH603133 2019-04-30 -0.302460 SZ300760 2019-04-30 -0.126383 -``Model`` module can make predictions, please refer to `Model `_. +``Interday Model`` module can make predictions, please refer to `Interday Model: Model Training & Prediction `_. Backtest Result ------------------ @@ -110,4 +110,4 @@ The backtest results are in the following form: Reference ============== -To know more about ``Intraday Trading``, please refer to `Backtest API <../reference/api.html>`_. +To know more about ``Intraday Trading``, please refer to `Intraday Trading <../reference/api.html#module-qlib.contrib.evaluate>`_. diff --git a/docs/component/data.rst b/docs/component/data.rst index a0e9a8398..6ebd94700 100644 --- a/docs/component/data.rst +++ b/docs/component/data.rst @@ -124,9 +124,9 @@ Feature - `ExpressionOps` `ExpressionOps` will use operator for feature construction. - To know more about ``Operator``, please refer to `Operator API <../reference/api.html>`_. + To know more about ``Operator``, please refer to `Operator API <../reference/api.html#module-qlib.data.ops>`_. -To know more about ``Feature``, please refer to `Feature API <../reference/api.html>`_. +To know more about ``Feature``, please refer to `Feature API <../reference/api.html#module-qlib.data.base>`_. Filter ------------------- @@ -142,25 +142,25 @@ Filter - `cross-sectional features filter` : rule_expression = '$rank($close)<10' - `time-sequence features filter`: rule_expression = '$Ref($close, 3)>100' -To know more about ``Filter``, please refer to `Filter API <../reference/api.html>`_. +To know more about ``Filter``, please refer to `Filter API <../reference/api.html#module-qlib.data.filter>`_. -API +Reference ------------- -To know more about ``Data API``, please refer to `Data API <../reference/api.html>`_. +To know more about ``Data API``, please refer to `Data API <../reference/api.html#data>`_. Data Handler ================= -Users can use ``Data Handler`` in an automatic workflow by ``Estimator``, refer to `Estimator `_ for more details. +Users can use ``Data Handler`` in an automatic workflow by ``Estimator``, refer to `Estimator: Workflow Management `_ for more details. Also, ``Data Handler`` can be used as an independent module, by which users can easily preprocess data(standardization, remove NaN, etc.) and build datasets. It is a subclass of ``qlib.contrib.estimator.handler.BaseDataHandler``, which provides some interfaces as follows. Base Class & Interface ---------------------- -Qlib provides a base class `qlib.contrib.estimator.BaseDataHandler <../reference/api.html#class-qlib.contrib.estimator.BaseDataHandler>`_, which provides the following interfaces: +Qlib provides a base class `qlib.contrib.estimator.BaseDataHandler <../reference/api.html#qlib.contrib.estimator.handler.BaseDataHandler>`_, which provides the following interfaces: - `setup_feature` Implement the interface to load the data features. @@ -204,7 +204,7 @@ Example ``Data Handler`` can be run with ``estimator`` by modifying the configuration file, and can also be used as a single module. -Know more about how to run ``Data Handler`` with ``estimator``, please refer to `Estimator `_. +Know more about how to run ``Data Handler`` with ``Estimator``, please refer to `Estimator: Workflow Management `_ Qlib provides implemented data handler `QLibDataHandlerClose`. The following example shows how to run `QLibDataHandlerV1` as a single module. @@ -243,14 +243,14 @@ Qlib provides implemented data handler `QLibDataHandlerClose`. The following exa print(x_train, y_train, x_validate, y_validate, x_test, y_test) -.. note:: (x_train, y_train, x_validate, y_validate, x_test, y_test) can be used as arguments for the ``fit``, ``predict``, and ``score`` methods of the 'Model' , please refer to `Model `_. +.. note:: (x_train, y_train, x_validate, y_validate, x_test, y_test) can be used as arguments for the `fit`, `predic``, and `score` methods of the ``Interday Model`` , please refer to `Model `_. Also, the above example has been given in ``examples.estimator.train_backtest_analyze.ipynb``. API --------- -To know more about ``Data Handler``, please refer to `Data Handler API <../reference/api.html#handler>`_. +To know more about ``Data Handler``, please refer to `Data Handler API <../reference/api.html#module-qlib.contrib.estimator.handler>`_. Cache ========== @@ -336,5 +336,3 @@ We've specially designed a file structure to manage data and cache, please refer - .index : an assorted index file recording the line index of all calendars - ... - -.. TODO: refer to paper diff --git a/docs/component/estimator.rst b/docs/component/estimator.rst index c8c0896ed..fdef92996 100644 --- a/docs/component/estimator.rst +++ b/docs/component/estimator.rst @@ -279,7 +279,7 @@ Qlib supports custom models, but it must be a subclass of the `qlib.contrib.mode The class `SomeModel` should be in the module `custom_model`, and ``Qlib`` could parse the `module_path` to load the class. -To know more about ``Model``, please refer to `Model `_. +To know more about ``Interday Model``, please refer to `Interday Model: Training & Prediction `_. Data Section ----------------- @@ -552,7 +552,7 @@ Users can specify `backtest` through a config file, for example: Backtest initial cash, integer type. The `account` in `strategy` section is deprecated. It only works when `account` is not set in `backtest` section. It will be overridden by `account` in the `backtest` section. The default value is 1e9. - `deal_price` - Order transaction price field, str type, the default value is vwap. + Order transaction price field, str type, the default value is close. - `min_cost` Min transaction cost, float type, the default value is 5. @@ -586,7 +586,7 @@ Experiment Result Form of Experimental Result ---------------------------- -The result of the experiment is also the result of the ``Interdat Trading(Backtest)``, please refer to `Interday Trading `_. +The result of the experiment is also the result of the ``Intraday Trading(Backtest)``, please refer to `Intraday Trading: Model&Strategy Testing `_. Get Experiment Result diff --git a/docs/component/model.rst b/docs/component/model.rst index ad01b7eee..36a77b1ca 100644 --- a/docs/component/model.rst +++ b/docs/component/model.rst @@ -6,7 +6,7 @@ Interday Model: Model Training & Prediction Introduction =================== -``Interday Model`` is designed to make the `prediction score` about stocks. Users can use the ``Interday Model`` in an automatic workflow by ``Estimator``, please refer to `Estimator `_. +``Interday Model`` is designed to make the `prediction score` about stocks. Users can use the ``Interday Model`` in an automatic workflow by ``Estimator``, please refer to `Estimator: Workflow Management `_. Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Model`` can be used as an independent module also. @@ -48,7 +48,7 @@ The base class provides the following interfaces: .. note:: - The number and names of the columns are determined by the data handler, please refer to `Data Handler `_ and `Estimator Data `_. + The number and names of the columns are determined by the data handler, please refer to `Data Handler `_ and `Estimator Data Section `_. - `y_train`, pd.DataFrame type, train label The following example explains the value of `y_train`: @@ -117,7 +117,7 @@ Example ``Qlib`` provides ``LightGBM`` and ``DNN`` models as the baseline, the following steps show how to run`` LightGBM`` as an independent module. -- Initialize ``Qlib`` with `qlib.init` first, please refer to `initialization <../start/initialization.html>`_. +- Initialize ``Qlib`` with `qlib.init` first, please refer to `Initialization <../start/initialization.html>`_. - Run the following code to get the `prediction score` `pred_score` .. code-block:: Python @@ -157,7 +157,6 @@ Example "num_threads": 20, } # use default model - # custom Model, refer to: TODO: Model API url model = LGBModel(**MODEL_CONFIG) model.fit(x_train, y_train, x_validate, y_validate) _pred = model.predict(x_test) diff --git a/docs/component/report.rst b/docs/component/report.rst index 11d765b33..55658a3d0 100644 --- a/docs/component/report.rst +++ b/docs/component/report.rst @@ -100,7 +100,7 @@ Graphical Result - Axis Y: - `ic` The `Pearson correlation coefficient` series between `label` and `prediction score`. - In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. Please refer to `Data API Featrue `_ for more details. + In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. Please refer to `Data Featrue `_ for more details. - `rank_ic` The `Spearman's rank correlation coefficient` series between `label` and `prediction score`. diff --git a/docs/component/strategy.rst b/docs/component/strategy.rst index c3aa88303..c0ee687ce 100644 --- a/docs/component/strategy.rst +++ b/docs/component/strategy.rst @@ -7,7 +7,7 @@ Interday Strategy: Portfolio Management Introduction =================== -``Interday Strategy`` is designed to adopt different trading strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Interday Model``. Users can use the ``Interday Strategy`` in an automatic workflow by ``Estimator``, please refer to `Estimator `_. +``Interday Strategy`` is designed to adopt different trading strategies, which means that users can adopt different algorithms to generate investment portfolios based on the prediction scores of the ``Interday Model``. Users can use the ``Interday Strategy`` in an automatic workflow by ``Estimator``, please refer to `Estimator: Workflow Management `_. Because the components in ``Qlib`` are designed in a loosely-coupled way, ``Interday Strategy`` can be used as an independent module also. @@ -95,11 +95,13 @@ Usage & Example "limit_threshold": 0.095, "account": 100000000, "benchmark": BENCHMARK, - "deal_price": "vwap", + "deal_price": "close", + "open_cost": 0.0005, + "close_cost": 0.0015, + "min_cost": 5, + } - # use default strategy - # custom Strategy, refer to: TODO: Strategy API url strategy = TopkDropoutStrategy(**STRATEGY_CONFIG) # pred_score is the `prediction score` output by Model @@ -115,4 +117,4 @@ To know more about ``Intraday Trading``, please refer to `Intraday Trading: Mode Reference =================== -To know more about ``Interday Strategy``, please refer to `Strategy API <../reference/api.html>`_. +To know more about ``Interday Strategy``, please refer to `Strategy API <../reference/api.html#module-qlib.contrib.strategy.strategy>`_. diff --git a/docs/introduction/quick.rst b/docs/introduction/quick.rst index d91a8fe7e..df4b84062 100644 --- a/docs/introduction/quick.rst +++ b/docs/introduction/quick.rst @@ -44,7 +44,7 @@ Load and prepare data by running the following code: This dataset is created by public data collected by crawler scripts in ``scripts/data_collector/``, which have been released in the same repository. Users could create the same dataset with it. -To kown more about `prepare data`, please refer to `Data Preparation <../component/data.html>`_. +To kown more about `prepare data`, please refer to `Data Preparation <../component/data.html#data-preparation>`_. Auto Quant Research Workflow ==================================== @@ -60,7 +60,7 @@ Auto Quant Research Workflow - Estimator result - The result of ``Estimator`` is as follows, which is also the result of ``Interday Trading``. Please refer to please refer to `Interdat Trading <../component/backtest.html>`_. for more details about the result. + The result of ``Estimator`` is as follows, which is also the result of ``Intraday Trading``. Please refer to `Intraday Trading <../component/backtest.html>`_. for more details about the result. .. code-block:: python @@ -77,7 +77,7 @@ Auto Quant Research Workflow max_drawdown -0.075024 - To know more about `Estimator`, please refer to `Estimator <../component/estimator.html>`_. + To know more about `Estimator`, please refer to `Estimator: Workflow Management <../component/estimator.html>`_. - Graphical Reports Analysis: - Run ``examples/estimator/analyze_from_estimator.ipynb`` with jupyter notebook diff --git a/docs/start/initialization.rst b/docs/start/initialization.rst index e2b601880..5b50e1eba 100644 --- a/docs/start/initialization.rst +++ b/docs/start/initialization.rst @@ -15,7 +15,7 @@ Please follow the steps below to initialize ``Qlib``. .. code-block:: bash python scripts/get_data.py qlib_data_cn --target_dir ~/.qlib/qlib_data/cn_data - Please refer to `Raw Data <../component/data.html>`_ for more information about ``get_data.py``, + Please refer to `Data Preparation <../component/data.html#data-preparation>`_ for more information about `get_data.py`, - Initialize Qlib before calling other APIs: run following code in python. diff --git a/docs/start/integration.rst b/docs/start/integration.rst index 293de8e03..614c82457 100644 --- a/docs/start/integration.rst +++ b/docs/start/integration.rst @@ -63,7 +63,7 @@ The Custom models need to inherit `qlib.contrib.model.base.Model <../reference/a - Override the `predict` method - The parameters include the test features. - Return the `prediction score`. - - Please refer to `qlib.contrib.model.base.Model <../reference/api.html#module-qlib.contrib.model.base>`_ for the parameter types of the fit method. + - Please refer to `Model API <../reference/api.html#module-qlib.contrib.model.base>`_ for the parameter types of the fit method. - Code Example: In the following example, users need to use dnn to predict the label(such as `preds`) of test data `x_test` and return it. .. code-block:: Python @@ -143,4 +143,4 @@ Also, ``Model`` can also be tested as a single module. An example has been given Reference ===================== -To know more about ``Model``, please refer to `Interday Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.contrib.model.base>`_. +To know more about ``Interday Model``, please refer to `Interday Model: Model Training & Prediction <../component/model.html>`_ and `Model API <../reference/api.html#module-qlib.contrib.model.base>`_.