mirror of
https://github.com/microsoft/qlib.git
synced 2026-07-01 01:51:18 +08:00
3e79a088ef
* add crypto symbols collectors * add crypto data collector * add crypto symbols collectors * add crypto data collector * solver region and source problem * fix merge * fix merge * clean all cn information Co-authored-by: DefangCui <170007807@pku.edu.cn>
55 lines
1.6 KiB
Markdown
55 lines
1.6 KiB
Markdown
# Collect Crypto Data
|
|
|
|
> *Please pay **ATTENTION** that the data is collected from [Coingecko](https://www.coingecko.com/en/api) and the data might not be perfect. We recommend users to prepare their own data if they have high-quality dataset. For more information, users can refer to the [related document](https://qlib.readthedocs.io/en/latest/component/data.html#converting-csv-format-into-qlib-format)*
|
|
|
|
## Requirements
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Usage of the dataset
|
|
> *Crypto dateset only support Data retrieval function but not support backtest function due to the lack of OHLC data.*
|
|
|
|
## Collector Data
|
|
|
|
|
|
### Crypto Data
|
|
|
|
#### 1d from Coingecko
|
|
|
|
```bash
|
|
|
|
# download from https://api.coingecko.com/api/v3/
|
|
python collector.py download_data --source_dir ~/.qlib/crypto_data/source/1d --start 2015-01-01 --end 2021-11-30 --delay 1 --interval 1d
|
|
|
|
# normalize
|
|
python collector.py normalize_data --source_dir ~/.qlib/crypto_data/source/1d --normalize_dir ~/.qlib/crypto_data/source/1d_nor --interval 1d --date_field_name date
|
|
|
|
# dump data
|
|
cd qlib/scripts
|
|
python dump_bin.py dump_all --csv_path ~/.qlib/crypto_data/source/1d_nor --qlib_dir ~/.qlib/qlib_data/crypto_data --freq day --date_field_name date --include_fields prices,total_volumes,market_caps
|
|
|
|
```
|
|
|
|
### using data
|
|
|
|
```python
|
|
import qlib
|
|
from qlib.data import D
|
|
|
|
qlib.init(provider_uri="~/.qlib/qlib_data/crypto_data")
|
|
df = D.features(D.instruments(market="all"), ["$prices", "$total_volumes","$market_caps"], freq="day")
|
|
```
|
|
|
|
|
|
### Help
|
|
```bash
|
|
python collector.py collector_data --help
|
|
```
|
|
|
|
## Parameters
|
|
|
|
- interval: 1d
|
|
- delay: 1
|