mirror of
https://github.com/microsoft/qlib.git
synced 2026-07-12 23:36:54 +08:00
Add Crypto dataset from coingecko (#733)
* add crypto symbols collectors * add crypto data collector * add crypto symbols collectors * add crypto data collector * solver region and source problem * fix merge * fix merge * clean all cn information Co-authored-by: DefangCui <170007807@pku.edu.cn>
This commit is contained in:
54
scripts/data_collector/crypto/README.md
Normal file
54
scripts/data_collector/crypto/README.md
Normal file
@@ -0,0 +1,54 @@
|
||||
# Collect Crypto Data
|
||||
|
||||
> *Please pay **ATTENTION** that the data is collected from [Coingecko](https://www.coingecko.com/en/api) and the data might not be perfect. We recommend users to prepare their own data if they have high-quality dataset. For more information, users can refer to the [related document](https://qlib.readthedocs.io/en/latest/component/data.html#converting-csv-format-into-qlib-format)*
|
||||
|
||||
## Requirements
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Usage of the dataset
|
||||
> *Crypto dateset only support Data retrieval function but not support backtest function due to the lack of OHLC data.*
|
||||
|
||||
## Collector Data
|
||||
|
||||
|
||||
### Crypto Data
|
||||
|
||||
#### 1d from Coingecko
|
||||
|
||||
```bash
|
||||
|
||||
# download from https://api.coingecko.com/api/v3/
|
||||
python collector.py download_data --source_dir ~/.qlib/crypto_data/source/1d --start 2015-01-01 --end 2021-11-30 --delay 1 --interval 1d
|
||||
|
||||
# normalize
|
||||
python collector.py normalize_data --source_dir ~/.qlib/crypto_data/source/1d --normalize_dir ~/.qlib/crypto_data/source/1d_nor --interval 1d --date_field_name date
|
||||
|
||||
# dump data
|
||||
cd qlib/scripts
|
||||
python dump_bin.py dump_all --csv_path ~/.qlib/crypto_data/source/1d_nor --qlib_dir ~/.qlib/qlib_data/crypto_data --freq day --date_field_name date --include_fields prices,total_volumes,market_caps
|
||||
|
||||
```
|
||||
|
||||
### using data
|
||||
|
||||
```python
|
||||
import qlib
|
||||
from qlib.data import D
|
||||
|
||||
qlib.init(provider_uri="~/.qlib/qlib_data/crypto_data")
|
||||
df = D.features(D.instruments(market="all"), ["$prices", "$total_volumes","$market_caps"], freq="day")
|
||||
```
|
||||
|
||||
|
||||
### Help
|
||||
```bash
|
||||
python collector.py collector_data --help
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
- interval: 1d
|
||||
- delay: 1
|
||||
Reference in New Issue
Block a user