diff --git a/docs/_static/img/analysis/analysis_model_IC.png b/docs/_static/img/analysis/analysis_model_IC.png index 4769e55e6..41cc318c9 100644 Binary files a/docs/_static/img/analysis/analysis_model_IC.png and b/docs/_static/img/analysis/analysis_model_IC.png differ diff --git a/docs/_static/img/analysis/analysis_model_NDQ.png b/docs/_static/img/analysis/analysis_model_NDQ.png index 5b3430d71..6ba015302 100644 Binary files a/docs/_static/img/analysis/analysis_model_NDQ.png and b/docs/_static/img/analysis/analysis_model_NDQ.png differ diff --git a/docs/_static/img/analysis/analysis_model_cumulative_return.png b/docs/_static/img/analysis/analysis_model_cumulative_return.png index 84dd2cde3..75760a73b 100644 Binary files a/docs/_static/img/analysis/analysis_model_cumulative_return.png and b/docs/_static/img/analysis/analysis_model_cumulative_return.png differ diff --git a/docs/_static/img/analysis/analysis_model_long_short.png b/docs/_static/img/analysis/analysis_model_long_short.png index c3afa07ab..871e5324b 100644 Binary files a/docs/_static/img/analysis/analysis_model_long_short.png and b/docs/_static/img/analysis/analysis_model_long_short.png differ diff --git a/docs/_static/img/analysis/analysis_model_monthly_IC.png b/docs/_static/img/analysis/analysis_model_monthly_IC.png index 86b4235eb..bc8d13361 100644 Binary files a/docs/_static/img/analysis/analysis_model_monthly_IC.png and b/docs/_static/img/analysis/analysis_model_monthly_IC.png differ diff --git a/docs/component/report.rst b/docs/component/report.rst index 55658a3d0..e642488e1 100644 --- a/docs/component/report.rst +++ b/docs/component/report.rst @@ -108,35 +108,35 @@ Graphical Result .. image:: ../_static/img/analysis/score_ic.png -Usage of `analysis_position.cumulative_return` ----------------------------------------------- - -API -~~~~~~~~~~~~~~~~ - -.. automodule:: qlib.contrib.report.analysis_position.cumulative_return - :members: - -Graphical Result -~~~~~~~~~~~~~~~~~ - -.. note:: - - - Axis X: Trading day - - Axis Y: - - Above axis Y: `(((Ref($close, -1)/$close - 1) * weight).sum() / weight.sum()).cumsum()` - - Below axis Y: Daily weight sum - - In the **sell** graph, `y < 0` stands for profit; in other cases, `y > 0` stands for profit. - - In the **buy_minus_sell** graph, the **y** value of the **weight** graph at the bottom is `buy_weight + sell_weight`. - - In each graph, the **red line** in the histogram on the right represents the average. - -.. image:: ../_static/img/analysis/cumulative_return_buy.png - -.. image:: ../_static/img/analysis/cumulative_return_sell.png - -.. image:: ../_static/img/analysis/cumulative_return_buy_minus_sell.png - -.. image:: ../_static/img/analysis/cumulative_return_hold.png +.. Usage of `analysis_position.cumulative_return` +.. ---------------------------------------------- +.. +.. API +.. ~~~~~~~~~~~~~~~~ +.. +.. .. automodule:: qlib.contrib.report.analysis_position.cumulative_return +.. :members: +.. +.. Graphical Result +.. ~~~~~~~~~~~~~~~~~ +.. +.. .. note:: +.. +.. - Axis X: Trading day +.. - Axis Y: +.. - Above axis Y: `(((Ref($close, -1)/$close - 1) * weight).sum() / weight.sum()).cumsum()` +.. - Below axis Y: Daily weight sum +.. - In the **sell** graph, `y < 0` stands for profit; in other cases, `y > 0` stands for profit. +.. - In the **buy_minus_sell** graph, the **y** value of the **weight** graph at the bottom is `buy_weight + sell_weight`. +.. - In each graph, the **red line** in the histogram on the right represents the average. +.. +.. .. image:: ../_static/img/analysis/cumulative_return_buy.png +.. +.. .. image:: ../_static/img/analysis/cumulative_return_sell.png +.. +.. .. image:: ../_static/img/analysis/cumulative_return_buy_minus_sell.png +.. +.. .. image:: ../_static/img/analysis/cumulative_return_hold.png Usage of `analysis_position.risk_analysis` @@ -152,7 +152,7 @@ API Graphical Result ~~~~~~~~~~~~~~~~~ -.. note:: +.. note:: - general graphics - `std` @@ -178,10 +178,10 @@ Graphical Result The `Maximum Drawdown` of `CAR` (cumulative abnormal return) with cost. -.. image:: ../_static/img/analysis/risk_analysis_bar.png +.. image:: ../_static/img/analysis/risk_analysis_bar.png :align: center -.. note:: +.. note:: - annualized_return/max_drawdown/information_ratio/std graphics - Axis X: Trading days grouped by month @@ -220,42 +220,42 @@ Graphical Result .. image:: ../_static/img/analysis/risk_analysis_std.png :align: center - -Usage of `analysis_position.rank_label` ----------------------------------------------- - -API -~~~~~ - -.. automodule:: qlib.contrib.report.analysis_position.rank_label - :members: - - -Graphical Result -~~~~~~~~~~~~~~~~~ - -.. note:: - - - hold/sell/buy graphics: - - Axis X: Trading day - - Axis Y: - Average `ranking ratio`of `label` for stocks that is held/sold/bought on the trading day. - - In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. The `ranking ratio` can be formulated as follows. - .. math:: - - ranking\ ratio = \frac{Ascending\ Ranking\ of\ label}{Number\ of\ Stocks\ in\ the\ Portfolio} - -.. image:: ../_static/img/analysis/rank_label_hold.png - :align: center - -.. image:: ../_static/img/analysis/rank_label_buy.png - :align: center - -.. image:: ../_static/img/analysis/rank_label_sell.png - :align: center - - +.. +.. Usage of `analysis_position.rank_label` +.. ---------------------------------------------- +.. +.. API +.. ~~~~~ +.. +.. .. automodule:: qlib.contrib.report.analysis_position.rank_label +.. :members: +.. +.. +.. Graphical Result +.. ~~~~~~~~~~~~~~~~~ +.. +.. .. note:: +.. +.. - hold/sell/buy graphics: +.. - Axis X: Trading day +.. - Axis Y: +.. Average `ranking ratio`of `label` for stocks that is held/sold/bought on the trading day. +.. +.. In the above example, the `label` is formulated as `Ref($close, -1)/$close - 1`. The `ranking ratio` can be formulated as follows. +.. .. math:: +.. +.. ranking\ ratio = \frac{Ascending\ Ranking\ of\ label}{Number\ of\ Stocks\ in\ the\ Portfolio} +.. +.. .. image:: ../_static/img/analysis/rank_label_hold.png +.. :align: center +.. +.. .. image:: ../_static/img/analysis/rank_label_buy.png +.. :align: center +.. +.. .. image:: ../_static/img/analysis/rank_label_sell.png +.. :align: center +.. +.. Usage of `analysis_model.analysis_model_performance` ----------------------------------------------------- diff --git a/examples/estimator/analyze_from_estimator.ipynb b/examples/estimator/analyze_from_estimator.ipynb index b0dd8a82d..3b4e7b703 100644 --- a/examples/estimator/analyze_from_estimator.ipynb +++ b/examples/estimator/analyze_from_estimator.ipynb @@ -80,25 +80,6 @@ "positions = pickle.load(estimator_dir.joinpath('positions.pkl').open('rb'))" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# get label data from qlib" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from qlib.data import D\n", - "pred_df_dates = pred_df.index.get_level_values(level='datetime')\n", - "features_df = D.features(D.instruments(MARKET), ['Ref($close, -1)/$close - 1'], pred_df_dates.min(), pred_df_dates.max())\n", - "features_df.columns = ['label']" - ] - }, { "cell_type": "markdown", "metadata": {}, @@ -112,7 +93,9 @@ "metadata": {}, "outputs": [], "source": [ - "from qlib.contrib.report import analysis_model, analysis_position" + "from qlib.data import D\n", + "from qlib.contrib.report import analysis_model, analysis_position\n", + "pred_df_dates = pred_df.index.get_level_values(level='datetime')" ] }, { @@ -122,6 +105,16 @@ "## analysis position" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "stock_ret = D.features(D.instruments(MARKET), ['Ref($close, -1)/$close - 1'], pred_df_dates.min(), pred_df_dates.max())\n", + "stock_ret.columns = ['label']" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -138,41 +131,6 @@ "analysis_position.report_graph(report_normal_df)" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### score IC" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "pred_label = pd.concat([features_df, pred_df], axis=1, sort=True).reindex(features_df.index)\n", - "analysis_position.score_ic_graph(pred_label)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### cumulative return" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "scrolled": false - }, - "outputs": [], - "source": [ - "analysis_position.cumulative_return_graph(positions, report_normal_df, features_df)" - ] - }, { "cell_type": "markdown", "metadata": {}, @@ -183,9 +141,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "scrolled": false - }, + "metadata": {}, "outputs": [], "source": [ "analysis_position.risk_analysis_graph(analysis_df, report_normal_df)" @@ -195,7 +151,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### rank label" + "## analysis model" ] }, { @@ -204,14 +160,25 @@ "metadata": {}, "outputs": [], "source": [ - "analysis_position.rank_label_graph(positions, features_df, pred_df_dates.min(), pred_df_dates.max())" + "label_df = D.features(D.instruments(MARKET), ['Ref($close, -2)/Ref($close, -1) - 1'], pred_df_dates.min(), pred_df_dates.max())\n", + "label_df.columns = ['label']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## analysis model" + "### score IC" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pred_label = pd.concat([label_df, pred_df], axis=1, sort=True).reindex(label_df.index)\n", + "analysis_position.score_ic_graph(pred_label)" ] }, { @@ -224,9 +191,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "scrolled": false - }, + "metadata": {}, "outputs": [], "source": [ "analysis_model.model_performance_graph(pred_label)" diff --git a/examples/train_backtest_analyze.ipynb b/examples/train_backtest_analyze.ipynb index b80996b54..6944ca617 100644 --- a/examples/train_backtest_analyze.ipynb +++ b/examples/train_backtest_analyze.ipynb @@ -62,7 +62,6 @@ "cell_type": "code", "execution_count": null, "metadata": { - "scrolled": true, "tags": [] }, "outputs": [], @@ -195,7 +194,19 @@ "metadata": {}, "outputs": [], "source": [ - "from qlib.contrib.report import analysis_model, analysis_position" + "from qlib.contrib.report import analysis_model, analysis_position\n", + "from qlib.data import D\n", + "pred_df_dates = pred_score.index.get_level_values(level='datetime')\n", + "report_normal_df = report_normal\n", + "positions = positions_normal\n", + "pred_df = pred_score" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## analysis position" ] }, { @@ -204,18 +215,8 @@ "metadata": {}, "outputs": [], "source": [ - "# get label data\n", - "from qlib.data import D\n", - "pred_df_dates = pred_score.index.get_level_values(level='datetime')\n", - "features_df = D.features(D.instruments(MARKET), ['Ref($close, -1)/$close - 1'], pred_df_dates.min(), pred_df_dates.max())\n", - "features_df.columns = ['label']" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## analysis position" + "stock_ret = D.features(D.instruments(MARKET), ['Ref($close, -1)/$close - 1'], pred_df_dates.min(), pred_df_dates.max())\n", + "stock_ret.columns = ['label']" ] }, { @@ -231,7 +232,40 @@ "metadata": {}, "outputs": [], "source": [ - "analysis_position.report_graph(report_normal)" + "analysis_position.report_graph(report_normal_df)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### risk analysis" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "analysis_position.risk_analysis_graph(analysis_df, report_normal_df)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## analysis model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "label_df = D.features(D.instruments(MARKET), ['Ref($close, -2)/Ref($close, -1) - 1'], pred_df_dates.min(), pred_df_dates.max())\n", + "label_df.columns = ['label']" ] }, { @@ -247,69 +281,10 @@ "metadata": {}, "outputs": [], "source": [ - "pred_label = pd.concat([features_df, pred_score], axis=1, sort=True).reindex(features_df.index)\n", + "pred_label = pd.concat([label_df, pred_df], axis=1, sort=True).reindex(label_df.index)\n", "analysis_position.score_ic_graph(pred_label)" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### cumulative return" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "scrolled": false - }, - "outputs": [], - "source": [ - "analysis_position.cumulative_return_graph(positions_normal, report_normal, features_df)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### risk analysis" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "scrolled": false - }, - "outputs": [], - "source": [ - "analysis_position.risk_analysis_graph(analysis_df, report_normal)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### rank label" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "analysis_position.rank_label_graph(positions_normal, features_df, pred_df_dates.min(), pred_df_dates.max())" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## analysis model" - ] - }, { "cell_type": "markdown", "metadata": {}, @@ -320,9 +295,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "scrolled": false - }, + "metadata": {}, "outputs": [], "source": [ "analysis_model.model_performance_graph(pred_label)" @@ -344,8 +317,7 @@ "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.5" + "pygments_lexer": "ipython3" }, "toc": { "base_numbering": 1, diff --git a/qlib/__init__.py b/qlib/__init__.py index a80c71643..df00ecfc1 100644 --- a/qlib/__init__.py +++ b/qlib/__init__.py @@ -2,7 +2,7 @@ # Licensed under the MIT License. -__version__ = "0.5.0" +__version__ = "0.5.0.dev" import os import copy diff --git a/qlib/contrib/report/analysis_model/analysis_model_performance.py b/qlib/contrib/report/analysis_model/analysis_model_performance.py index b879a467d..96ee6bea7 100644 --- a/qlib/contrib/report/analysis_model/analysis_model_performance.py +++ b/qlib/contrib/report/analysis_model/analysis_model_performance.py @@ -264,17 +264,18 @@ def model_performance_graph( ) -> [list, tuple]: """Model performance - :param pred_label: index is **pd.MultiIndex**, index name is **[instrument, datetime]**; columns names is **[score, label]** + :param pred_label: index is **pd.MultiIndex**, index name is **[instrument, datetime]**; columns names is **[score, + label]**. It is usually same as the label of model training(e.g. "Ref($close, -2)/Ref($close, -1) - 1") .. code-block:: python instrument datetime score label SH600004 2017-12-11 -0.013502 -0.013502 - 2017-12-12 -0.072367 -0.072367 - 2017-12-13 -0.068605 -0.068605 - 2017-12-14 0.012440 0.012440 - 2017-12-15 -0.102778 -0.102778 + 2017-12-12 -0.072367 -0.072367 + 2017-12-13 -0.068605 -0.068605 + 2017-12-14 0.012440 0.012440 + 2017-12-15 -0.102778 -0.102778 :param lag: `pred.groupby(level='instrument')['score'].shift(lag)`. It will be only used in the auto-correlation computing. diff --git a/setup.py b/setup.py index c975697c4..6715dfcf8 100644 --- a/setup.py +++ b/setup.py @@ -12,7 +12,7 @@ from setuptools import find_packages, setup, Extension NAME = "qlib" DESCRIPTION = "A Quantitative-research Platform" REQUIRES_PYTHON = ">=3.5.0" -VERSION = "0.5.0" +VERSION = "0.5.0.dev" # Detect Cython try: