mirror of
https://github.com/microsoft/qlib.git
synced 2026-07-19 02:14:33 +08:00
Compare commits
7 Commits
migrate_gy
...
5314ab1dc7
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5314ab1dc7 | ||
|
|
ac3fe9476f | ||
|
|
66c36226aa | ||
|
|
bb7ab1cf14 | ||
|
|
3dc5a7d299 | ||
|
|
7d66e4b788 | ||
|
|
213eb6c2cd |
13
CHANGELOG.md
13
CHANGELOG.md
@@ -0,0 +1,13 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
## [0.9.8](https://github.com/microsoft/qlib/compare/v0.9.7...v0.9.8) (2025-11-10)
|
||||||
|
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
* download orderbook data error ([#1990](https://github.com/microsoft/qlib/issues/1990)) ([136b2dd](https://github.com/microsoft/qlib/commit/136b2ddf9a16e4106d62b8d1336a56273a8abef0))
|
||||||
|
* **macd:** remove extra division by close in DEA calculation to ensure dimension consistency ([#2046](https://github.com/microsoft/qlib/issues/2046)) ([66c3622](https://github.com/microsoft/qlib/commit/66c36226aafceabe497e5967f67921e5d3c9d497))
|
||||||
|
* replace deprecated pandas fillna(method=) with ffill()/bfill() ([#1987](https://github.com/microsoft/qlib/issues/1987)) ([7095e75](https://github.com/microsoft/qlib/commit/7095e755fa57e011f0483d24b45fc5bd5a4deaf8))
|
||||||
|
* spelling errors ([#1996](https://github.com/microsoft/qlib/issues/1996)) ([f26b341](https://github.com/microsoft/qlib/commit/f26b3417363410531dbbb39e425bce6cf05528a1))
|
||||||
|
* the bug when auto_mount=True ([#2009](https://github.com/microsoft/qlib/issues/2009)) ([213eb6c](https://github.com/microsoft/qlib/commit/213eb6c2cd12342b6ec98f21300217e1659f3d58))
|
||||||
|
* typo in integration documentation: 'userd' -> 'used' ([#2034](https://github.com/microsoft/qlib/issues/2034)) ([3dc5a7d](https://github.com/microsoft/qlib/commit/3dc5a7d299074f0fa45a4b7bb50ab446a8824a32))
|
||||||
|
|||||||
2
Makefile
2
Makefile
@@ -113,7 +113,7 @@ dev: prerequisite all
|
|||||||
|
|
||||||
# Check lint with black.
|
# Check lint with black.
|
||||||
black:
|
black:
|
||||||
black . -l 120 --check --diff
|
black . -l 120 --check --diff --exclude qlib/_version.py
|
||||||
|
|
||||||
# Check code folder with pylint.
|
# Check code folder with pylint.
|
||||||
# TODO: These problems we will solve in the future. Important among them are: W0221, W0223, W0237, E1102
|
# TODO: These problems we will solve in the future. Important among them are: W0221, W0223, W0237, E1102
|
||||||
|
|||||||
@@ -324,7 +324,7 @@ We recommend users to prepare their own data if they have a high-quality dataset
|
|||||||
```
|
```
|
||||||
2. Start a new Docker container
|
2. Start a new Docker container
|
||||||
```bash
|
```bash
|
||||||
docker run -it --name <container name> -v <Mounted local directory>:/app qlib_image_stable
|
docker run -it --name <container name> -v <Mounted local directory>:/app pyqlib/qlib_image_stable:stable
|
||||||
```
|
```
|
||||||
3. At this point you are in the docker environment and can run the qlib scripts. An example:
|
3. At this point you are in the docker environment and can run the qlib scripts. An example:
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -42,7 +42,7 @@ Example
|
|||||||
|
|
||||||
.. math::
|
.. math::
|
||||||
|
|
||||||
DEA = \frac{EMA(DIF, 9)}{CLOSE}
|
DEA = EMA(DIF, 9)
|
||||||
|
|
||||||
Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
||||||
|
|
||||||
@@ -51,7 +51,7 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
|||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
>> from qlib.data.dataset.loader import QlibDataLoader
|
>> from qlib.data.dataset.loader import QlibDataLoader
|
||||||
>> MACD_EXP = '(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close'
|
>> MACD_EXP = '2 * ((EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9))'
|
||||||
>> fields = [MACD_EXP] # MACD
|
>> fields = [MACD_EXP] # MACD
|
||||||
>> names = ['MACD']
|
>> names = ['MACD']
|
||||||
>> labels = ['Ref($close, -2)/Ref($close, -1) - 1'] # label
|
>> labels = ['Ref($close, -2)/Ref($close, -1) - 1'] # label
|
||||||
@@ -66,17 +66,17 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
|||||||
feature label
|
feature label
|
||||||
MACD LABEL
|
MACD LABEL
|
||||||
datetime instrument
|
datetime instrument
|
||||||
2010-01-04 SH600000 -0.011547 -0.019672
|
2010-01-04 SH600000 0.008781 -0.019672
|
||||||
SH600004 0.002745 -0.014721
|
SH600004 0.006699 -0.014721
|
||||||
SH600006 0.010133 0.002911
|
SH600006 0.005714 0.002911
|
||||||
SH600008 -0.001113 0.009818
|
SH600008 0.000798 0.009818
|
||||||
SH600009 0.025878 -0.017758
|
SH600009 0.017015 -0.017758
|
||||||
... ... ...
|
... ... ...
|
||||||
2017-12-29 SZ300124 0.007306 -0.005074
|
2017-12-29 SZ300124 0.015071 -0.005074
|
||||||
SZ300136 -0.013492 0.056352
|
SZ300136 -0.015466 0.056352
|
||||||
SZ300144 -0.000966 0.011853
|
SZ300144 0.013082 0.011853
|
||||||
SZ300251 0.004383 0.021739
|
SZ300251 -0.001026 0.021739
|
||||||
SZ300315 -0.030557 0.012455
|
SZ300315 -0.007559 0.012455
|
||||||
|
|
||||||
Reference
|
Reference
|
||||||
=========
|
=========
|
||||||
|
|||||||
@@ -129,7 +129,7 @@ For example, it looks quite long and complicated:
|
|||||||
|
|
||||||
|
|
||||||
But using string is not the only way to implement the expression. You can also implement expression by code.
|
But using string is not the only way to implement the expression. You can also implement expression by code.
|
||||||
Here is an exmaple which does the same thing as above examples.
|
Here is an example which does the same thing as above examples.
|
||||||
|
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|||||||
@@ -71,7 +71,7 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html#
|
|||||||
)
|
)
|
||||||
|
|
||||||
- Override the `predict` method
|
- Override the `predict` method
|
||||||
- The parameters must include the parameter `dataset`, which will be userd to get the test dataset.
|
- The parameters must include the parameter `dataset`, which will be used to get the test dataset.
|
||||||
- Return the `prediction score`.
|
- Return the `prediction score`.
|
||||||
- Please refer to `Model API <../reference/api.html#module-qlib.model.base>`_ for the parameter types of the fit method.
|
- Please refer to `Model API <../reference/api.html#module-qlib.model.base>`_ for the parameter types of the fit method.
|
||||||
- Code Example: In the following example, users need to use `LightGBM` to predict the label(such as `preds`) of test data `x_test` and return it.
|
- Code Example: In the following example, users need to use `LightGBM` to predict the label(such as `preds`) of test data `x_test` and return it.
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ dependencies = [
|
|||||||
"pymongo",
|
"pymongo",
|
||||||
"loguru",
|
"loguru",
|
||||||
"lightgbm",
|
"lightgbm",
|
||||||
"gymnasium<=0.26.2",
|
"gym",
|
||||||
"cvxpy",
|
"cvxpy",
|
||||||
"joblib",
|
"joblib",
|
||||||
"matplotlib",
|
"matplotlib",
|
||||||
@@ -117,3 +117,4 @@ qrun = "qlib.cli.run:run"
|
|||||||
[tool.setuptools_scm]
|
[tool.setuptools_scm]
|
||||||
local_scheme = "no-local-version"
|
local_scheme = "no-local-version"
|
||||||
version_scheme = "guess-next-dev"
|
version_scheme = "guess-next-dev"
|
||||||
|
write_to = "qlib/_version.py"
|
||||||
|
|||||||
@@ -4,7 +4,10 @@ from pathlib import Path
|
|||||||
|
|
||||||
from setuptools_scm import get_version
|
from setuptools_scm import get_version
|
||||||
|
|
||||||
__version__ = get_version(root="..", relative_to=__file__)
|
try:
|
||||||
|
from ._version import version as __version__
|
||||||
|
except ImportError:
|
||||||
|
__version__ = get_version(root="..", relative_to=__file__)
|
||||||
__version__bak = __version__ # This version is backup for QlibConfig.reset_qlib_version
|
__version__bak = __version__ # This version is backup for QlibConfig.reset_qlib_version
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
@@ -140,7 +143,10 @@ def _mount_nfs_uri(provider_uri, mount_path, auto_mount: bool = False):
|
|||||||
_command_log = [line for line in _command_log if _remote_uri in line]
|
_command_log = [line for line in _command_log if _remote_uri in line]
|
||||||
if len(_command_log) > 0:
|
if len(_command_log) > 0:
|
||||||
for _c in _command_log:
|
for _c in _command_log:
|
||||||
_temp_mount = _c.decode("utf-8").split(" ")[2]
|
if isinstance(_c, str):
|
||||||
|
_temp_mount = _c.split(" ")[2]
|
||||||
|
else:
|
||||||
|
_temp_mount = _c.decode("utf-8").split(" ")[2]
|
||||||
_temp_mount = _temp_mount[:-1] if _temp_mount.endswith("/") else _temp_mount
|
_temp_mount = _temp_mount[:-1] if _temp_mount.endswith("/") else _temp_mount
|
||||||
if _temp_mount == _mount_path:
|
if _temp_mount == _mount_path:
|
||||||
_is_mount = True
|
_is_mount = True
|
||||||
|
|||||||
@@ -5,9 +5,9 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from typing import Any, Generic, TypeVar
|
from typing import Any, Generic, TypeVar
|
||||||
|
|
||||||
import gymnasium as gym
|
import gym
|
||||||
import numpy as np
|
import numpy as np
|
||||||
from gymnasium import spaces
|
from gym import spaces
|
||||||
|
|
||||||
from qlib.typehint import final
|
from qlib.typehint import final
|
||||||
from .simulator import ActType, StateType
|
from .simulator import ActType, StateType
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ from typing import Any, List, Optional, cast
|
|||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
from gymnasium import spaces
|
from gym import spaces
|
||||||
|
|
||||||
from qlib.constant import EPS
|
from qlib.constant import EPS
|
||||||
from qlib.rl.data.base import ProcessedDataProvider
|
from qlib.rl.data.base import ProcessedDataProvider
|
||||||
|
|||||||
@@ -6,11 +6,11 @@ from __future__ import annotations
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Dict, Generator, Iterable, Optional, OrderedDict, Tuple, cast
|
from typing import Any, Dict, Generator, Iterable, Optional, OrderedDict, Tuple, cast
|
||||||
|
|
||||||
import gymnasium as gym
|
import gym
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import torch
|
import torch
|
||||||
import torch.nn as nn
|
import torch.nn as nn
|
||||||
from gymnasium.spaces import Discrete
|
from gym.spaces import Discrete
|
||||||
from tianshou.data import Batch, ReplayBuffer, to_torch
|
from tianshou.data import Batch, ReplayBuffer, to_torch
|
||||||
from tianshou.policy import BasePolicy, PPOPolicy, DQNPolicy
|
from tianshou.policy import BasePolicy, PPOPolicy, DQNPolicy
|
||||||
|
|
||||||
|
|||||||
@@ -6,8 +6,8 @@ from __future__ import annotations
|
|||||||
import weakref
|
import weakref
|
||||||
from typing import Any, Callable, cast, Dict, Generic, Iterable, Iterator, Optional, Tuple
|
from typing import Any, Callable, cast, Dict, Generic, Iterable, Iterator, Optional, Tuple
|
||||||
|
|
||||||
import gymnasium as gym
|
import gym
|
||||||
from gymnasium import Space
|
from gym import Space
|
||||||
|
|
||||||
from qlib.rl.aux_info import AuxiliaryInfoCollector
|
from qlib.rl.aux_info import AuxiliaryInfoCollector
|
||||||
from qlib.rl.interpreter import ActionInterpreter, ObsType, PolicyActType, StateInterpreter
|
from qlib.rl.interpreter import ActionInterpreter, ObsType, PolicyActType, StateInterpreter
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ import warnings
|
|||||||
from contextlib import contextmanager
|
from contextlib import contextmanager
|
||||||
from typing import Any, Callable, Dict, Generator, List, Optional, Set, Tuple, Type, Union, cast
|
from typing import Any, Callable, Dict, Generator, List, Optional, Set, Tuple, Type, Union, cast
|
||||||
|
|
||||||
import gymnasium as gym
|
import gym
|
||||||
import numpy as np
|
import numpy as np
|
||||||
from tianshou.env import BaseVectorEnv, DummyVectorEnv, ShmemVectorEnv, SubprocVectorEnv
|
from tianshou.env import BaseVectorEnv, DummyVectorEnv, ShmemVectorEnv, SubprocVectorEnv
|
||||||
|
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ class InfoCollector:
|
|||||||
"pymongo",
|
"pymongo",
|
||||||
"loguru",
|
"loguru",
|
||||||
"lightgbm",
|
"lightgbm",
|
||||||
"gymnasium",
|
"gym",
|
||||||
"cvxpy",
|
"cvxpy",
|
||||||
"joblib",
|
"joblib",
|
||||||
"matplotlib",
|
"matplotlib",
|
||||||
|
|||||||
@@ -82,7 +82,7 @@ def get_calendar_list(bench_code="CSI300") -> List[pd.Timestamp]:
|
|||||||
if bench_code.upper() == "ALL":
|
if bench_code.upper() == "ALL":
|
||||||
|
|
||||||
@deco_retry
|
@deco_retry
|
||||||
def _get_calendar(month):
|
def _get_calendar_from_month(month):
|
||||||
_cal = []
|
_cal = []
|
||||||
try:
|
try:
|
||||||
resp = requests.get(
|
resp = requests.get(
|
||||||
@@ -98,7 +98,7 @@ def get_calendar_list(bench_code="CSI300") -> List[pd.Timestamp]:
|
|||||||
month_range = pd.date_range(start="2000-01", end=pd.Timestamp.now() + pd.Timedelta(days=31), freq="M")
|
month_range = pd.date_range(start="2000-01", end=pd.Timestamp.now() + pd.Timedelta(days=31), freq="M")
|
||||||
calendar = []
|
calendar = []
|
||||||
for _m in month_range:
|
for _m in month_range:
|
||||||
cal = _get_calendar(_m.strftime("%Y-%m"))
|
cal = _get_calendar_from_month(_m.strftime("%Y-%m"))
|
||||||
if cal:
|
if cal:
|
||||||
calendar += cal
|
calendar += cal
|
||||||
calendar = list(filter(lambda x: x <= pd.Timestamp.now(), calendar))
|
calendar = list(filter(lambda x: x <= pd.Timestamp.now(), calendar))
|
||||||
|
|||||||
@@ -613,10 +613,6 @@ class YahooNormalize1min(YahooNormalize, ABC):
|
|||||||
def symbol_to_yahoo(self, symbol):
|
def symbol_to_yahoo(self, symbol):
|
||||||
raise NotImplementedError("rewrite symbol_to_yahoo")
|
raise NotImplementedError("rewrite symbol_to_yahoo")
|
||||||
|
|
||||||
@abc.abstractmethod
|
|
||||||
def _get_1d_calendar_list(self) -> Iterable[pd.Timestamp]:
|
|
||||||
raise NotImplementedError("rewrite _get_1d_calendar_list")
|
|
||||||
|
|
||||||
|
|
||||||
class YahooNormalizeUS:
|
class YahooNormalizeUS:
|
||||||
def _get_calendar_list(self) -> Iterable[pd.Timestamp]:
|
def _get_calendar_list(self) -> Iterable[pd.Timestamp]:
|
||||||
|
|||||||
10
setup.py
10
setup.py
@@ -2,22 +2,12 @@ import os
|
|||||||
|
|
||||||
import numpy
|
import numpy
|
||||||
from setuptools import Extension, setup
|
from setuptools import Extension, setup
|
||||||
from setuptools_scm import get_version
|
|
||||||
|
|
||||||
|
|
||||||
def read(rel_path: str) -> str:
|
|
||||||
here = os.path.abspath(os.path.dirname(__file__))
|
|
||||||
with open(os.path.join(here, rel_path), encoding="utf-8") as fp:
|
|
||||||
return fp.read()
|
|
||||||
|
|
||||||
|
|
||||||
NUMPY_INCLUDE = numpy.get_include()
|
NUMPY_INCLUDE = numpy.get_include()
|
||||||
|
|
||||||
|
|
||||||
VERSION = get_version(root=".", relative_to=__file__)
|
|
||||||
|
|
||||||
setup(
|
setup(
|
||||||
version=VERSION,
|
|
||||||
ext_modules=[
|
ext_modules=[
|
||||||
Extension(
|
Extension(
|
||||||
"qlib.data._libs.rolling",
|
"qlib.data._libs.rolling",
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
|
|
||||||
from collections import Counter
|
from collections import Counter
|
||||||
|
|
||||||
import gymnasium as gym
|
import gym
|
||||||
import numpy as np
|
import numpy as np
|
||||||
from tianshou.data import Batch, Collector
|
from tianshou.data import Batch, Collector
|
||||||
from tianshou.policy import BasePolicy
|
from tianshou.policy import BasePolicy
|
||||||
|
|||||||
@@ -7,10 +7,10 @@ import logging
|
|||||||
import re
|
import re
|
||||||
from typing import Any, Tuple
|
from typing import Any, Tuple
|
||||||
|
|
||||||
import gymnasium as gym
|
import gym
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
from gymnasium import spaces
|
from gym import spaces
|
||||||
from tianshou.data import Collector, Batch
|
from tianshou.data import Collector, Batch
|
||||||
from tianshou.policy import BasePolicy
|
from tianshou.policy import BasePolicy
|
||||||
|
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import pytest
|
|||||||
|
|
||||||
import torch
|
import torch
|
||||||
import torch.nn as nn
|
import torch.nn as nn
|
||||||
from gymnasium import spaces
|
from gym import spaces
|
||||||
from tianshou.policy import PPOPolicy
|
from tianshou.policy import PPOPolicy
|
||||||
|
|
||||||
from qlib.config import C
|
from qlib.config import C
|
||||||
|
|||||||
Reference in New Issue
Block a user