* Fixed pandas FutureWarning
`FutureWarning: Passing a set as an indexer is deprecated and will raise in a future version. Use a list instead.`
* fixed another pandas FutureWarning
```
scripts/data_collector/index.py:228: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
new_df = new_df.append(_tmp_df, sort=False)
```
* fixed more pandas futurewarnings
* feat: download ibovespa index historic composition
ibovespa(ibov) is the largest index in Brazil's stocks exchange.
The br_index folder has support for downloading new companies for the current index composition.
And has support, as well, for downloading companies from historic composition of ibov index.
Partially resolves issue #956
* fix: typo error instead of end_date, it was written end_ate
* feat: adds support for downloading stocks historic prices from Brazil's stocks exchange (B3)
Together with commit c2f933 it resolves issue #956
* fix: code formatted with black.
* wip: Creating code logic for brazils stock market data normalization
* docs: brazils stock market data normalization code documentation
* fix: code formatted the with black
* docs: fixed typo
* docs: more info about python version used to generate requirements.txt file
* docs: added BeautifulSoup requirements
* feat: removed debug prints
* feat: added ibov_index_composition variable as a class attribute of IBOVIndex
* feat: added increment to generate the four month period used by the ibov index
* refactor: Added get_instruments() method inside utils.py for better code usability.
Message in the PR request to understand the context of the change
In the course of reviewing this PR we found two issues.
1. there are multiple places where the get_instruments() method is used,
and we feel that scripts.index.py is the best place for the
get_instruments() method to go.
2. data_collector.utils has some very generic stuff put inside it.
* refactor: improve brazils stocks download speed
The reason to use retry=2 is due to the fact that
Yahoo Finance unfortunately does not keep track of the majority
of Brazilian stocks.
Therefore, the decorator deco_retry with retry argument
set to 5 will keep trying to get the stock data 5 times,
which makes the code to download Brazilians stocks very slow.
In future, this may change, but for now
I suggest to leave retry argument to 1 or 2 in
order to improve download speed.
In order to achieve this code logic an argument called retry_config
was added into YahooCollectorBR1d and YahooCollectorBR1min
* fix: added __main__ at the bottom of the script
* refactor: changed interface inside each index
Using partial as `fire.Fire(partial(get_instruments, market_index="br_index" ))`
will make the interface easier for the user to execute the script.
Then all the collector.py CLI in each folder can remove a redundant arguments.
* refactor: implemented class interface retry into YahooCollectorBR
* docs: added BR as a possible region into the documentation
* refactor: make retry attribute part of the interface
This way we don't have to use hasattr to access the retry attribute as previously done
* add period ops class
* black format
* add pit data read
* fix bug in period ops
* update ops runnable
* update PIT test example
* black format
* update PIT test
* update tets_PIT
* update code format
* add check_feature_exist
* black format
* optimize the PIT Algorithm
* fix bug
* update example
* update test_PIT name
* add pit collector
* black format
* fix bugs
* fix try
* fix bug & add dump_pit.py
* Successfully run and understand PIT
* Add some docs and remove a bug
* mv crypto collector
* black format
* Run succesfully after merging master
* Pass test and fix code
* remove useless PIT code
* fix PYlint
* Rename
Co-authored-by: Young <afe.young@gmail.com>
* Fix high-freq data name from `yahoo_cn_1min` to `cn_data_1min`
* re-format example data names using `qlib_{region}_{feq}`, e.g. qlib_cn_1d
* re-format example data names using `{region}_{feq}`, e.g. us_1d and cn_1min
* keep using for 1day data, and change 1min data to
* updated readme of yahoo collector where region parameter was incorrect
* changes
update readme of yahoo collector where region parameter was incorrect
* update readme of yahoo collector
update readme of yahoo collector where region parameter was incorrect
* updated changes
* updated readme of cn1d data
Co-authored-by: Gaurav Chauhan01/HO/Analytics/General <Gaurav.Chauhan01@bajajallianz.in>
