qlib/examples/rl

This folder contains a simple example of how to run Qlib RL. It contains:

.
├── experiment_config
│   ├── backtest       # Backtest config
│   └── training       # Training config
├── README.md          # Readme (the current file)
└── scripts            # Scripts for data pre-processing

Data preparation

Use AzCopy to download data:

azcopy copy https://qlibpublic.blob.core.windows.net/data/rl/qlib_rl_example_data ./ --recursive
mv qlib_rl_example_data data

The downloaded data will be placed at ./data. The original data are in data/csv. To create all data needed by the case, run:

bash scripts/data_pipeline.sh

After the execution finishes, the data/ directory should be like:

data
├── backtest_orders.csv
├── bin
├── csv
├── pickle
├── pickle_dataframe
└── training_order_split

Run training

Run:

python -m qlib.rl.contrib.train_onpolicy --config_path ./experiment_config/training/config.yml

After training, checkpoints will be stored under checkpoints/.

Run backtest

python -m qlib.rl.contrib.backtest --config_path ./experiment_config/backtest/config.yml

The backtest workflow will use the trained model in checkpoints/. The backtest summary can be found in outputs/.

Others

The RL module is designed in a loosely-coupled way. Currently, RL examples are integrated with concrete business logic. But the core part of RL is much simpler than what you see. To demonstrate the simple core of RL, a dedicated notebook for RL without business loss is created.