DDG-DA paper code (#743)

* Merge data selection to main * Update trainer for reweighter * Typos fixed. * update data selection interface * successfully run exp after refactor some interface * data selection share handler & trainer * fix meta model time series bug * fix online workflow set_uri bug * fix set_uri bug * updawte ds docs and delay trainer bug * docs * resume reweighter * add reweighting result * fix qlib model import * make recorder more friendly * fix experiment workflow bug * commit for merging master incase of conflictions * Successful run DDG-DA with a single command * remove unused code * asdd more docs * Update README.md * Update & fix some bugs. * Update configuration & remove debug functions * Update README.md * Modfify horizon from code rather than yaml * Update performance in README.md * fix part comments * Remove unfinished TCTS. * Fix some details. * Update meta docs * Update README.md of the benchmarks_dynamic * Update README.md files * Add README.md to the rolling_benchmark baseline. * Refine the docs and link * Rename README.md in benchmarks_dynamic. * Remove comments. * auto download data Co-authored-by: wendili-cs <wendili.academic@qq.com> Co-authored-by: demon143 <785696300@qq.com>
2026-07-10 06:20:57 +08:00 · 2022-01-10 16:52:37 +08:00
parent 184ce34a34
commit cf35562e84
52 changed files with 2441 additions and 456 deletions
--- a/qlib/model/meta/task.py
+++ b/qlib/model/meta/task.py
@@ -0,0 +1,53 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
+import abc
+from typing import Union, List, Tuple
+
+from qlib.data.dataset import Dataset
+from ...utils import init_instance_by_config
+
+
+class MetaTask:
+    """
+    A single meta-task, a meta-dataset contains a list of them.
+    It serves as a component as in MetaDatasetDS
+
+    The data processing is different
+    - the processed input may be different between training and testing
+        - When training, the X, y, X_test, y_test in training tasks are necessary (# PROC_MODE_FULL #)
+                                                but not necessary in test tasks. (# PROC_MODE_TEST #)
+        - When the meta model can be transferred into other dataset, only meta_info is necessary  (# PROC_MODE_TRANSFER #)
+    """
+
+    PROC_MODE_FULL = "full"
+    PROC_MODE_TEST = "test"
+    PROC_MODE_TRANSFER = "transfer"
+
+    def __init__(self, task: dict, meta_info: object, mode: str = PROC_MODE_FULL):
+        """
+        The `__init__` func is responsible for
+        - store the task
+        - store the origin input data for
+        - process the input data for meta data
+
+        Parameters
+        ----------
+        task : dict
+            the task to be enhanced by meta model
+
+        meta_info : object
+            the input for meta model
+        """
+        self.task = task
+        self.meta_info = meta_info  # the original meta input information, it will be processed later
+        self.mode = mode
+
+    def get_dataset(self) -> Dataset:
+        return init_instance_by_config(self.task["dataset"], accept_types=Dataset)
+
+    def get_meta_input(self) -> object:
+        """
+        Return the **processed** meta_info
+        """
+        return self.meta_info