diff --git a/docs/_static/img/Task-Gen-Recorder-Collector.svg b/docs/_static/img/Task-Gen-Recorder-Collector.svg new file mode 100644 index 000000000..51602c12b --- /dev/null +++ b/docs/_static/img/Task-Gen-Recorder-Collector.svg @@ -0,0 +1,4 @@ + + + +
Collector.process_list:
  • qlib.model.ens.ensemble.AverageEnsemble
  • qlib.model.ens.ensemble.RollingEnsemble    
Collector.process_list:...
Combine and Collect Data
Combine and Collect Data
Task Genratation
Task Genratation
 Task_{GBDT}
 Task_{NN}
Rolling Generator
Rolling Generat...
Genrated Tasks
Genrated Tasks
Task_{GBDT ,t...
Task_{GBDT ,t...
Task_{GBDT ,T...

{"model": { 
       class: "...GBDT"
   },
   "dataset": {
        ....
        "kwargs" {
             segments: {
                test: [....]
                 ....
             }
        }
   }
}

{"model": {...
time
time
Rolling
Generator
Rolling...
Genrated Tasks
Genrated Tasks
 Task_{NN,t1}
 Task_{NN,t2}
time
time
 Task_{NN,T}
 Record_{NN,t1}
pred.pkl
pred.pkl
 Pred_{avg,t1}
 Record_{NN,t2}
pred.pkl
pred.pkl
 Record_{NN,T}
pred.pkl
pred.pkl
 Record_{GBDT,t1}
pred.pkl
pred.pkl
 Record_{GBDT,t2}
pred.pkl
pred.pkl
 Record_{GBDT,T}
pred.pkl
pred.pkl
 Pred_{avg,t2}
 Pred_{avg,T}
qlib.model.ens.ensemble.RollingEnsemble
qlib.model.ens.ensemble.RollingEnsemble
Final Result
Final Result
...
...
...
...
...
...
...
...
...
...
...
...
qlib.model.ens.ensemble.AverageEnsemble
qlib.model.ens.ensemble.AverageEnsemble
qlib.model.ens.group.Group
qlib.model.ens.group.Group
+ Group
It will categorize data into groups
+ Group...
+ Reduce
It calls ensemble to combine the data
+ Reduce...
'score':  {
   {
           : <pd.DataFrame>
   }
}
'score':  {...
 t1
Model Training
Model Training
...
...
qlib.model.trainer.Trainer
qlib.model.trainer.Trainer
...
...
...
...
...
...
...
...Viewer does not support full SVG 1.1 \ No newline at end of file diff --git a/docs/advanced/task_management.rst b/docs/advanced/task_management.rst index 56a3137f9..611cdf136 100644 --- a/docs/advanced/task_management.rst +++ b/docs/advanced/task_management.rst @@ -11,7 +11,11 @@ Introduction The `Workflow <../component/introduction.html>`_ part introduces how to run research workflow in a loosely-coupled way. But it can only execute one ``task`` when you use ``qrun``. To automatically generate and execute different tasks, ``Task Management`` provides a whole process including `Task Generating`_, `Task Storing`_, `Task Training`_ and `Task Collecting`_. -With this module, users can run their ``task`` automatically at different periods, in different losses, or even by different models. +With this module, users can run their ``task`` automatically at different periods, in different losses, or even by different models.The processes of task generation, model training and combine and collect data are shown in the following figure. + +
+ +
This whole process can be used in `Online Serving <../component/online.html>`_. @@ -74,6 +78,8 @@ If you do not want to use ``Task Manager`` to manage tasks, then use TrainerR to Task Collecting =============== +Before collecting model training results, you need to use the ``qlib.init`` to specify the path of mlruns. + To collect the results of ``task`` after training, ``Qlib`` provides `Collector <../reference/api.html#Collector>`_, `Group <../reference/api.html#Group>`_ and `Ensemble <../reference/api.html#Ensemble>`_ to collect the results in a readable, expandable and loosely-coupled way. `Collector <../reference/api.html#Collector>`_ can collect objects from everywhere and process them such as merging, grouping, averaging and so on. It has 2 step action including ``collect`` (collect anything in a dict) and ``process_collect`` (process collected dict). @@ -82,8 +88,10 @@ To collect the results of ``task`` after training, ``Qlib`` provides `Collector For example: {(A,B,C1): object, (A,B,C2): object} ---``group``---> {(A,B): {C1: object, C2: object}} ---``reduce``---> {(A,B): object} `Ensemble <../reference/api.html#Ensemble>`_ can merge the objects in an ensemble. -For example: {C1: object, C2: object} ---``Ensemble``---> object +For example: {C1: object, C2: object} ---``Ensemble``---> object. +You can set the ensembles you want in the ``Collector``'s process_list. +Common ensembles include ``AverageEnsemble`` and ``RollingEnsemble``. Average ensemble is used to ensemble the results of different models in the same time period. Rollingensemble is used to ensemble the results of different models in the same time period So the hierarchy is ``Collector``'s second step corresponds to ``Group``. And ``Group``'s second step correspond to ``Ensemble``. -For more information, please see `Collector <../reference/api.html#Collector>`_, `Group <../reference/api.html#Group>`_ and `Ensemble <../reference/api.html#Ensemble>`_, or the `example `_. \ No newline at end of file +For more information, please see `Collector <../reference/api.html#Collector>`_, `Group <../reference/api.html#Group>`_ and `Ensemble <../reference/api.html#Ensemble>`_, or the `example `_.