mirror of
https://github.com/microsoft/qlib.git
synced 2026-06-29 00:51:19 +08:00
Compare commits
2 Commits
release-pl
...
migrate_gy
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2276552c1f | ||
|
|
9eee6d33b0 |
14
CHANGELOG.md
14
CHANGELOG.md
@@ -1,14 +0,0 @@
|
||||
# Changelog
|
||||
|
||||
## [0.9.8](https://github.com/microsoft/qlib/compare/v0.9.7...v0.9.8) (2025-11-13)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* download orderbook data error ([#1990](https://github.com/microsoft/qlib/issues/1990)) ([136b2dd](https://github.com/microsoft/qlib/commit/136b2ddf9a16e4106d62b8d1336a56273a8abef0))
|
||||
* **gbdt:** correct dtrain assignment in finetune() to use Dataset instead of tuple ([#2049](https://github.com/microsoft/qlib/issues/2049)) ([2b41782](https://github.com/microsoft/qlib/commit/2b41782f0cfb81e8cc065f2915b215758a7838ef))
|
||||
* **macd:** remove extra division by close in DEA calculation to ensure dimension consistency ([#2046](https://github.com/microsoft/qlib/issues/2046)) ([66c3622](https://github.com/microsoft/qlib/commit/66c36226aafceabe497e5967f67921e5d3c9d497))
|
||||
* replace deprecated pandas fillna(method=) with ffill()/bfill() ([#1987](https://github.com/microsoft/qlib/issues/1987)) ([7095e75](https://github.com/microsoft/qlib/commit/7095e755fa57e011f0483d24b45fc5bd5a4deaf8))
|
||||
* spelling errors ([#1996](https://github.com/microsoft/qlib/issues/1996)) ([f26b341](https://github.com/microsoft/qlib/commit/f26b3417363410531dbbb39e425bce6cf05528a1))
|
||||
* the bug when auto_mount=True ([#2009](https://github.com/microsoft/qlib/issues/2009)) ([213eb6c](https://github.com/microsoft/qlib/commit/213eb6c2cd12342b6ec98f21300217e1659f3d58))
|
||||
* typo in integration documentation: 'userd' -> 'used' ([#2034](https://github.com/microsoft/qlib/issues/2034)) ([3dc5a7d](https://github.com/microsoft/qlib/commit/3dc5a7d299074f0fa45a4b7bb50ab446a8824a32))
|
||||
|
||||
2
Makefile
2
Makefile
@@ -113,7 +113,7 @@ dev: prerequisite all
|
||||
|
||||
# Check lint with black.
|
||||
black:
|
||||
black . -l 120 --check --diff --exclude qlib/_version.py
|
||||
black . -l 120 --check --diff
|
||||
|
||||
# Check code folder with pylint.
|
||||
# TODO: These problems we will solve in the future. Important among them are: W0221, W0223, W0237, E1102
|
||||
|
||||
@@ -324,7 +324,7 @@ We recommend users to prepare their own data if they have a high-quality dataset
|
||||
```
|
||||
2. Start a new Docker container
|
||||
```bash
|
||||
docker run -it --name <container name> -v <Mounted local directory>:/app pyqlib/qlib_image_stable:stable
|
||||
docker run -it --name <container name> -v <Mounted local directory>:/app qlib_image_stable
|
||||
```
|
||||
3. At this point you are in the docker environment and can run the qlib scripts. An example:
|
||||
```bash
|
||||
|
||||
@@ -42,7 +42,7 @@ Example
|
||||
|
||||
.. math::
|
||||
|
||||
DEA = EMA(DIF, 9)
|
||||
DEA = \frac{EMA(DIF, 9)}{CLOSE}
|
||||
|
||||
Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
||||
|
||||
@@ -51,7 +51,7 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
||||
.. code-block:: python
|
||||
|
||||
>> from qlib.data.dataset.loader import QlibDataLoader
|
||||
>> MACD_EXP = '2 * ((EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9))'
|
||||
>> MACD_EXP = '(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close'
|
||||
>> fields = [MACD_EXP] # MACD
|
||||
>> names = ['MACD']
|
||||
>> labels = ['Ref($close, -2)/Ref($close, -1) - 1'] # label
|
||||
@@ -66,17 +66,17 @@ Users can use ``Data Handler`` to build formulaic alphas `MACD` in qlib:
|
||||
feature label
|
||||
MACD LABEL
|
||||
datetime instrument
|
||||
2010-01-04 SH600000 0.008781 -0.019672
|
||||
SH600004 0.006699 -0.014721
|
||||
SH600006 0.005714 0.002911
|
||||
SH600008 0.000798 0.009818
|
||||
SH600009 0.017015 -0.017758
|
||||
2010-01-04 SH600000 -0.011547 -0.019672
|
||||
SH600004 0.002745 -0.014721
|
||||
SH600006 0.010133 0.002911
|
||||
SH600008 -0.001113 0.009818
|
||||
SH600009 0.025878 -0.017758
|
||||
... ... ...
|
||||
2017-12-29 SZ300124 0.015071 -0.005074
|
||||
SZ300136 -0.015466 0.056352
|
||||
SZ300144 0.013082 0.011853
|
||||
SZ300251 -0.001026 0.021739
|
||||
SZ300315 -0.007559 0.012455
|
||||
2017-12-29 SZ300124 0.007306 -0.005074
|
||||
SZ300136 -0.013492 0.056352
|
||||
SZ300144 -0.000966 0.011853
|
||||
SZ300251 0.004383 0.021739
|
||||
SZ300315 -0.030557 0.012455
|
||||
|
||||
Reference
|
||||
=========
|
||||
|
||||
@@ -129,7 +129,7 @@ For example, it looks quite long and complicated:
|
||||
|
||||
|
||||
But using string is not the only way to implement the expression. You can also implement expression by code.
|
||||
Here is an example which does the same thing as above examples.
|
||||
Here is an exmaple which does the same thing as above examples.
|
||||
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@@ -71,7 +71,7 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html#
|
||||
)
|
||||
|
||||
- Override the `predict` method
|
||||
- The parameters must include the parameter `dataset`, which will be used to get the test dataset.
|
||||
- The parameters must include the parameter `dataset`, which will be userd to get the test dataset.
|
||||
- Return the `prediction score`.
|
||||
- Please refer to `Model API <../reference/api.html#module-qlib.model.base>`_ for the parameter types of the fit method.
|
||||
- Code Example: In the following example, users need to use `LightGBM` to predict the label(such as `preds`) of test data `x_test` and return it.
|
||||
|
||||
@@ -45,7 +45,7 @@ dependencies = [
|
||||
"pymongo",
|
||||
"loguru",
|
||||
"lightgbm",
|
||||
"gym",
|
||||
"gymnasium<=0.26.2",
|
||||
"cvxpy",
|
||||
"joblib",
|
||||
"matplotlib",
|
||||
@@ -117,4 +117,3 @@ qrun = "qlib.cli.run:run"
|
||||
[tool.setuptools_scm]
|
||||
local_scheme = "no-local-version"
|
||||
version_scheme = "guess-next-dev"
|
||||
write_to = "qlib/_version.py"
|
||||
|
||||
@@ -4,10 +4,7 @@ from pathlib import Path
|
||||
|
||||
from setuptools_scm import get_version
|
||||
|
||||
try:
|
||||
from ._version import version as __version__
|
||||
except ImportError:
|
||||
__version__ = get_version(root="..", relative_to=__file__)
|
||||
__version__ = get_version(root="..", relative_to=__file__)
|
||||
__version__bak = __version__ # This version is backup for QlibConfig.reset_qlib_version
|
||||
import logging
|
||||
import os
|
||||
@@ -143,10 +140,7 @@ def _mount_nfs_uri(provider_uri, mount_path, auto_mount: bool = False):
|
||||
_command_log = [line for line in _command_log if _remote_uri in line]
|
||||
if len(_command_log) > 0:
|
||||
for _c in _command_log:
|
||||
if isinstance(_c, str):
|
||||
_temp_mount = _c.split(" ")[2]
|
||||
else:
|
||||
_temp_mount = _c.decode("utf-8").split(" ")[2]
|
||||
_temp_mount = _c.decode("utf-8").split(" ")[2]
|
||||
_temp_mount = _temp_mount[:-1] if _temp_mount.endswith("/") else _temp_mount
|
||||
if _temp_mount == _mount_path:
|
||||
_is_mount = True
|
||||
|
||||
@@ -51,7 +51,7 @@ class LGBModel(ModelFT, LightGBMFInt):
|
||||
w = reweighter.reweight(df)
|
||||
else:
|
||||
raise ValueError("Unsupported reweighter type.")
|
||||
ds_l.append((lgb.Dataset(x.values, label=y, weight=w, free_raw_data=False), key))
|
||||
ds_l.append((lgb.Dataset(x.values, label=y, weight=w), key))
|
||||
return ds_l
|
||||
|
||||
def fit(
|
||||
@@ -109,10 +109,8 @@ class LGBModel(ModelFT, LightGBMFInt):
|
||||
verbose level
|
||||
"""
|
||||
# Based on existing model and finetune by train more rounds
|
||||
ds_l = self._prepare_data(dataset, reweighter)
|
||||
dtrain, _ = ds_l[0]
|
||||
|
||||
if dtrain.construct().num_data() == 0:
|
||||
dtrain, _ = self._prepare_data(dataset, reweighter) # pylint: disable=W0632
|
||||
if dtrain.empty:
|
||||
raise ValueError("Empty data from dataset, please check your dataset config.")
|
||||
verbose_eval_callback = lgb.log_evaluation(period=verbose_eval)
|
||||
self.model = lgb.train(
|
||||
|
||||
@@ -5,9 +5,9 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any, Generic, TypeVar
|
||||
|
||||
import gym
|
||||
import gymnasium as gym
|
||||
import numpy as np
|
||||
from gym import spaces
|
||||
from gymnasium import spaces
|
||||
|
||||
from qlib.typehint import final
|
||||
from .simulator import ActType, StateType
|
||||
|
||||
@@ -8,7 +8,7 @@ from typing import Any, List, Optional, cast
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from gym import spaces
|
||||
from gymnasium import spaces
|
||||
|
||||
from qlib.constant import EPS
|
||||
from qlib.rl.data.base import ProcessedDataProvider
|
||||
|
||||
@@ -6,11 +6,11 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Generator, Iterable, Optional, OrderedDict, Tuple, cast
|
||||
|
||||
import gym
|
||||
import gymnasium as gym
|
||||
import numpy as np
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
from gym.spaces import Discrete
|
||||
from gymnasium.spaces import Discrete
|
||||
from tianshou.data import Batch, ReplayBuffer, to_torch
|
||||
from tianshou.policy import BasePolicy, PPOPolicy, DQNPolicy
|
||||
|
||||
|
||||
@@ -6,8 +6,8 @@ from __future__ import annotations
|
||||
import weakref
|
||||
from typing import Any, Callable, cast, Dict, Generic, Iterable, Iterator, Optional, Tuple
|
||||
|
||||
import gym
|
||||
from gym import Space
|
||||
import gymnasium as gym
|
||||
from gymnasium import Space
|
||||
|
||||
from qlib.rl.aux_info import AuxiliaryInfoCollector
|
||||
from qlib.rl.interpreter import ActionInterpreter, ObsType, PolicyActType, StateInterpreter
|
||||
|
||||
@@ -13,7 +13,7 @@ import warnings
|
||||
from contextlib import contextmanager
|
||||
from typing import Any, Callable, Dict, Generator, List, Optional, Set, Tuple, Type, Union, cast
|
||||
|
||||
import gym
|
||||
import gymnasium as gym
|
||||
import numpy as np
|
||||
from tianshou.env import BaseVectorEnv, DummyVectorEnv, ShmemVectorEnv, SubprocVectorEnv
|
||||
|
||||
|
||||
@@ -45,7 +45,7 @@ class InfoCollector:
|
||||
"pymongo",
|
||||
"loguru",
|
||||
"lightgbm",
|
||||
"gym",
|
||||
"gymnasium",
|
||||
"cvxpy",
|
||||
"joblib",
|
||||
"matplotlib",
|
||||
|
||||
@@ -82,7 +82,7 @@ def get_calendar_list(bench_code="CSI300") -> List[pd.Timestamp]:
|
||||
if bench_code.upper() == "ALL":
|
||||
|
||||
@deco_retry
|
||||
def _get_calendar_from_month(month):
|
||||
def _get_calendar(month):
|
||||
_cal = []
|
||||
try:
|
||||
resp = requests.get(
|
||||
@@ -98,7 +98,7 @@ def get_calendar_list(bench_code="CSI300") -> List[pd.Timestamp]:
|
||||
month_range = pd.date_range(start="2000-01", end=pd.Timestamp.now() + pd.Timedelta(days=31), freq="M")
|
||||
calendar = []
|
||||
for _m in month_range:
|
||||
cal = _get_calendar_from_month(_m.strftime("%Y-%m"))
|
||||
cal = _get_calendar(_m.strftime("%Y-%m"))
|
||||
if cal:
|
||||
calendar += cal
|
||||
calendar = list(filter(lambda x: x <= pd.Timestamp.now(), calendar))
|
||||
|
||||
@@ -613,6 +613,10 @@ class YahooNormalize1min(YahooNormalize, ABC):
|
||||
def symbol_to_yahoo(self, symbol):
|
||||
raise NotImplementedError("rewrite symbol_to_yahoo")
|
||||
|
||||
@abc.abstractmethod
|
||||
def _get_1d_calendar_list(self) -> Iterable[pd.Timestamp]:
|
||||
raise NotImplementedError("rewrite _get_1d_calendar_list")
|
||||
|
||||
|
||||
class YahooNormalizeUS:
|
||||
def _get_calendar_list(self) -> Iterable[pd.Timestamp]:
|
||||
|
||||
10
setup.py
10
setup.py
@@ -2,12 +2,22 @@ import os
|
||||
|
||||
import numpy
|
||||
from setuptools import Extension, setup
|
||||
from setuptools_scm import get_version
|
||||
|
||||
|
||||
def read(rel_path: str) -> str:
|
||||
here = os.path.abspath(os.path.dirname(__file__))
|
||||
with open(os.path.join(here, rel_path), encoding="utf-8") as fp:
|
||||
return fp.read()
|
||||
|
||||
|
||||
NUMPY_INCLUDE = numpy.get_include()
|
||||
|
||||
|
||||
VERSION = get_version(root=".", relative_to=__file__)
|
||||
|
||||
setup(
|
||||
version=VERSION,
|
||||
ext_modules=[
|
||||
Extension(
|
||||
"qlib.data._libs.rolling",
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
|
||||
from collections import Counter
|
||||
|
||||
import gym
|
||||
import gymnasium as gym
|
||||
import numpy as np
|
||||
from tianshou.data import Batch, Collector
|
||||
from tianshou.policy import BasePolicy
|
||||
|
||||
@@ -7,10 +7,10 @@ import logging
|
||||
import re
|
||||
from typing import Any, Tuple
|
||||
|
||||
import gym
|
||||
import gymnasium as gym
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from gym import spaces
|
||||
from gymnasium import spaces
|
||||
from tianshou.data import Collector, Batch
|
||||
from tianshou.policy import BasePolicy
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ import pytest
|
||||
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
from gym import spaces
|
||||
from gymnasium import spaces
|
||||
from tianshou.policy import PPOPolicy
|
||||
|
||||
from qlib.config import C
|
||||
|
||||
Reference in New Issue
Block a user