diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 000000000..bc00efde3 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,8 @@ +__pycache__ +*.pyc +*.pyo +*.pyd +.Python +.env +.git + diff --git a/.readthedocs.yaml b/.readthedocs.yaml index 71b29a227..cc939ff80 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -9,7 +9,7 @@ version: 2 build: os: ubuntu-22.04 tools: - python: "3.7" + python: "3.8" # Build documentation in the docs/ directory with Sphinx sphinx: diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 000000000..7eb8c315c --- /dev/null +++ b/Dockerfile @@ -0,0 +1,31 @@ +FROM continuumio/miniconda3:latest + +WORKDIR /qlib + +COPY . . + +RUN apt-get update && \ + apt-get install -y build-essential + +RUN conda create --name qlib_env python=3.8 -y +RUN echo "conda activate qlib_env" >> ~/.bashrc +ENV PATH /opt/conda/envs/qlib_env/bin:$PATH + +RUN python -m pip install --upgrade pip + +RUN python -m pip install numpy==1.23.5 +RUN python -m pip install pandas==1.5.3 +RUN python -m pip install importlib-metadata==5.2.0 +RUN python -m pip install "cloudpickle<3" +RUN python -m pip install scikit-learn==1.3.2 + +RUN python -m pip install cython packaging tables matplotlib statsmodels +RUN python -m pip install pybind11 cvxpy + +ARG IS_STABLE="yes" + +RUN if [ "$IS_STABLE" = "yes" ]; then \ + python -m pip install pyqlib; \ + else \ + python setup.py install; \ + fi diff --git a/README.md b/README.md index 465d8331f..d79aa24cd 100644 --- a/README.md +++ b/README.md @@ -292,6 +292,38 @@ We recommend users to prepare their own data if they have a high-quality dataset ``` --> +## Docker images +1. Pulling a docker image from a docker hub repository + ```bash + docker pull pyqlib/qlib_image_stable:stable + ``` +2. Start a new Docker container + ```bash + docker run -it --name -v :/app qlib_image_stable + ``` +3. At this point you are in the docker environment and can run the qlib scripts. An example: + ```bash + >>> python scripts/get_data.py qlib_data --name qlib_data_simple --target_dir ~/.qlib/qlib_data/cn_data --interval 1d --region cn + >>> python qlib/workflow/cli.py examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml + ``` +4. Exit the container + ```bash + >>> exit + ``` +5. Restart the container + ```bash + docker start -i -a + ``` +6. Stop the container + ```bash + docker stop + ``` +7. Delete the container + ```bash + docker rm + ``` +8. If you want to know more information, please refer to the [documentation](https://qlib.readthedocs.io/en/latest/developer/how_to_build_image.html). + ## Auto Quant Research Workflow Qlib provides a tool named `qrun` to run the whole workflow automatically (including building dataset, training models, backtest and evaluation). You can start an auto quant research workflow and have a graphical reports analysis according to the following steps: diff --git a/build_docker_image.sh b/build_docker_image.sh new file mode 100644 index 000000000..4acf963a2 --- /dev/null +++ b/build_docker_image.sh @@ -0,0 +1,31 @@ +#!/bin/bash + +docker_user="your_dockerhub_username" + +read -p "Do you want to build the nightly version of the qlib image? (default is stable) (yes/no): " answer; +answer=$(echo "$answer" | tr '[:upper:]' '[:lower:]') + +if [ "$answer" = "yes" ]; then + # Build the nightly version of the qlib image + docker build --build-arg IS_STABLE=no -t qlib_image -f ./Dockerfile . + image_tag="nightly" +else + # Build the stable version of the qlib image + docker build -t qlib_image -f ./Dockerfile . + image_tag="stable" +fi + +read -p "Is it uploaded to docker hub? (default is no) (yes/no): " answer; +answer=$(echo "$answer" | tr '[:upper:]' '[:lower:]') + +if [ "$answer" = "yes" ]; then + # Log in to Docker Hub + # If you are a new docker hub user, please verify your email address before proceeding with this step. + docker login + # Tag the Docker image + docker tag qlib_image "$docker_user/qlib_image:$image_tag" + # Push the Docker image to Docker Hub + docker push "$docker_user/qlib_image:$image_tag" +else + echo "Not uploaded to docker hub." +fi diff --git a/docs/conf.py b/docs/conf.py index 442c89da2..f1377c140 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -123,7 +123,6 @@ html_logo = "_static/img/logo/1.png" html_theme_options = { "logo_only": True, "collapse_navigation": False, - "display_version": False, "navigation_depth": 4, } diff --git a/docs/developer/how_to_build_image.rst b/docs/developer/how_to_build_image.rst new file mode 100644 index 000000000..7f79c6fbb --- /dev/null +++ b/docs/developer/how_to_build_image.rst @@ -0,0 +1,81 @@ +.. _docker_image: + +================== +Build Docker Image +================== + +Dockerfile +========== + +There is a **Dockerfile** file in the root directory of the project from which you can build the docker image. There are two build methods in Dockerfile to choose from. +When executing the build command, use the ``--build-arg`` parameter to control the image version. The ``--build-arg`` parameter defaults to ``yes``, which builds the ``stable`` version of the qlib image. + +1.For the ``stable`` version, use ``pip install pyqlib`` to build the qlib image. + +.. code-block:: bash + + docker build --build-arg IS_STABLE=yes -t -f ./Dockerfile . + +.. code-block:: bash + + docker build -t -f ./Dockerfile . + +2. For the ``nightly`` version, use current source code to build the qlib image. + +.. code-block:: bash + + docker build --build-arg IS_STABLE=no -t -f ./Dockerfile . + +Auto build of qlib images +========================= + +1. There is a **build_docker_image.sh** file in the root directory of your project, which can be used to automatically build docker images and upload them to your docker hub repository(Optional, configuration required). + +.. code-block:: bash + + sh build_docker_image.sh + >>> Do you want to build the nightly version of the qlib image? (default is stable) (yes/no): + >>> Is it uploaded to docker hub? (default is no) (yes/no): + +2. If you want to upload the built image to your docker hub repository, you need to edit your **build_docker_image.sh** file first, fill in ``docker_user`` in the file, and then execute this file. + +How to use qlib images +====================== +1. Start a new Docker container + +.. code-block:: bash + + docker run -it --name -v :/app + +2. At this point you are in the docker environment and can run the qlib scripts. An example: + +.. code-block:: bash + + >>> python scripts/get_data.py qlib_data --name qlib_data_simple --target_dir ~/.qlib/qlib_data/cn_data --interval 1d --region cn + >>> python qlib/workflow/cli.py examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml + +3. Exit the container + +.. code-block:: bash + + >>> exit + +4. Restart the container + +.. code-block:: bash + + docker start -i -a + +5. Stop the container + +.. code-block:: bash + + docker stop -i -a + +6. Delete the container + +.. code-block:: bash + + docker rm + +7. For more information on using docker see the `docker documentation `_. diff --git a/docs/index.rst b/docs/index.rst index 3adf9049a..49596234a 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -61,6 +61,7 @@ Document Structure :caption: FOR DEVELOPERS: Code Standard & Development Guidance + How to build image .. toctree:: :maxdepth: 3 diff --git a/setup.py b/setup.py index a0dc9962c..bce3b812d 100644 --- a/setup.py +++ b/setup.py @@ -47,6 +47,10 @@ REQUIRED = [ "numpy>=1.12.0, <1.24", "pandas>=0.25.1", "scipy>=1.7.3", + # scs is a dependency package, + # and the latest version of scs: scs-3.2.4.post3.tar.gz causes the documentation build to fail, + # so we have temporarily limited the version of scs. + "scs<=3.2.4", "requests>=2.18.0", "sacred>=0.7.4", "python-socketio",