qiskit/.travis.yml

105 lines
3.1 KiB
YAML
Raw Normal View History

# This code is part of Qiskit.
2017-05-12 00:12:57 +08:00
#
# (C) Copyright IBM 2017.
#
# This code is licensed under the Apache License, Version 2.0. You may
# obtain a copy of this license in the LICENSE.txt file in the root directory
# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
#
# Any modifications or derivative works of this code must retain this
# copyright notice, and modified files need to carry a notice indicating
# that they have been altered from the originals.
2017-05-12 00:12:57 +08:00
Revise travis configuration, using cmake * Revise the travis configuration for using `cmake` for the several targets, and use "stages" instead of parallel jobs: * define three stages that are executed if the previous one suceeds: 1. "linter and pure python test": executes the linter and a test without compiling the binaries, with the idea of providing quick feedback for PRs. 2. "test": launch the test, including the compilation of binaries, under GNU/Linux Python 3.6 and 3.6; and osx Python 3.6. 3. "deploy doc and pypi": for the stable branch, deploy the docs to the landing page, and when using a specific commit message, build the GNU/Linux and osx wheels, uploading them to test.pypi. * use yaml anchors and definitions to avoid repeating code (and working around travis limitations). * Modify the `cmake``configuration to accomodate the stages flow: * allow conditional creation of compilation and QA targets, mainly for saving some time in some jobs. * move the tests to `cmake/tests.cmake`. * Update the tests: * add a `requires_qe_access` decorator that retrieves QE_TOKEN and QE_URL and appends them to the parameters in an unified manner. * add an environment variable `SKIP_ONLINE_TESTS` that allows to skip the tests that need network access. * replace `TRAVIS_FORK_PULL_REQUEST` with the previous two mechanisms, adding support for AppVeyor as well. * fix a problem with matplotlib under osx headless, effectively skipping `test_visualization.py` during the travis osx jobs. * Move Sphinx to `requirements-dev.txt`.
2018-02-13 05:11:28 +08:00
notifications:
email: false
2017-05-12 00:12:57 +08:00
cache:
pip: true
directories:
- .stestr
2017-05-11 00:44:30 +08:00
Revise travis configuration, using cmake * Revise the travis configuration for using `cmake` for the several targets, and use "stages" instead of parallel jobs: * define three stages that are executed if the previous one suceeds: 1. "linter and pure python test": executes the linter and a test without compiling the binaries, with the idea of providing quick feedback for PRs. 2. "test": launch the test, including the compilation of binaries, under GNU/Linux Python 3.6 and 3.6; and osx Python 3.6. 3. "deploy doc and pypi": for the stable branch, deploy the docs to the landing page, and when using a specific commit message, build the GNU/Linux and osx wheels, uploading them to test.pypi. * use yaml anchors and definitions to avoid repeating code (and working around travis limitations). * Modify the `cmake``configuration to accomodate the stages flow: * allow conditional creation of compilation and QA targets, mainly for saving some time in some jobs. * move the tests to `cmake/tests.cmake`. * Update the tests: * add a `requires_qe_access` decorator that retrieves QE_TOKEN and QE_URL and appends them to the parameters in an unified manner. * add an environment variable `SKIP_ONLINE_TESTS` that allows to skip the tests that need network access. * replace `TRAVIS_FORK_PULL_REQUEST` with the previous two mechanisms, adding support for AppVeyor as well. * fix a problem with matplotlib under osx headless, effectively skipping `test_visualization.py` during the travis osx jobs. * Move Sphinx to `requirements-dev.txt`.
2018-02-13 05:11:28 +08:00
stage_generic: &stage_generic
install:
# Install step for jobs that require compilation and qa.
- pip install -U -r requirements.txt -c constraints.txt
- pip install -U -r requirements-dev.txt coveralls -c constraints.txt
- pip install -c constraints.txt -e .
- pip install "qiskit-ibmq-provider" -c constraints.txt
Revise travis configuration, using cmake * Revise the travis configuration for using `cmake` for the several targets, and use "stages" instead of parallel jobs: * define three stages that are executed if the previous one suceeds: 1. "linter and pure python test": executes the linter and a test without compiling the binaries, with the idea of providing quick feedback for PRs. 2. "test": launch the test, including the compilation of binaries, under GNU/Linux Python 3.6 and 3.6; and osx Python 3.6. 3. "deploy doc and pypi": for the stable branch, deploy the docs to the landing page, and when using a specific commit message, build the GNU/Linux and osx wheels, uploading them to test.pypi. * use yaml anchors and definitions to avoid repeating code (and working around travis limitations). * Modify the `cmake``configuration to accomodate the stages flow: * allow conditional creation of compilation and QA targets, mainly for saving some time in some jobs. * move the tests to `cmake/tests.cmake`. * Update the tests: * add a `requires_qe_access` decorator that retrieves QE_TOKEN and QE_URL and appends them to the parameters in an unified manner. * add an environment variable `SKIP_ONLINE_TESTS` that allows to skip the tests that need network access. * replace `TRAVIS_FORK_PULL_REQUEST` with the previous two mechanisms, adding support for AppVeyor as well. * fix a problem with matplotlib under osx headless, effectively skipping `test_visualization.py` during the travis osx jobs. * Move Sphinx to `requirements-dev.txt`.
2018-02-13 05:11:28 +08:00
script:
# Compile the executables and run the tests.
- python setup.py build_ext --inplace
- export PYTHONHASHSEED=$(python -S -c "import random; print(random. randint(1, 4294967295))")
- echo "PYTHONHASHSEED=$PYTHONHASHSEED"
- stestr run
after_failure:
- python tools/report_ci_failure.py
Add python 3.8 support and deprecate python 3.5 (#3268) * Add python 3.8 support and deprecate python 3.5 This commit adds support for running terra under python 3.8. It does this by adding the trove classifiers to the package metadata and adding a test job for python 3.8 (only travis for now this can be expanded in the future when azure pipelines adds it to their image). At the same time this commit starts the deprecation window for python 3.5. Python 3.5 goes end of life by upstream python in Sept. 2020. We should give our users running with python 3.5 (which is about 10% of our users based on pypi data) fair notice that when upstream python stops supporting it we do as well. * Fix lint * Add back empty cache detection As part of the refactors the empty stestr timing cache removal code was removed from the travis config, but the cache wasn't removed. Adding the python 3.8 job causes a failure because travis has no cached timing data but still creates the empty directory which triggers mtreinish/stestr#266. This adds back the empty cache removal to workaround it so we can use the timing data for scheduling in future runs. * Add azure-pipelines python 3.8 jobs too * Fix python 3.8 dictionary keys changed error Starting in python 3.8 a new RuntimeError is raised, RuntimeError: dictionary keys changed during iteration. This is caused by modifying an iterator while looping over it. We were doing that in the optimize_swap_before_measure pass. This commit fixes this by wrapping the iterator in list() to make a copy of it for looping. This way we don't modify the contents of what we're iterating over in the pass. * Revert "Add azure-pipelines python 3.8 jobs too" The missing matplotlib wheels are blockers for windows and osx environment. Since we do not have the necessary dependencies installed in those ci envs to compile matplotlib from source. We'll rely on just travis for 3.8 testing until the matplotlib 3.2.0 release is pushed so we don't have to compile it. This reverts commit 40157621b135a03b291b93e5d9b801237a14c015. * Add skip for failing matplotlib test The image comparison tests which are quite flaky are failing with matplotlib compiled on 3.8. Looking at the image output from these failures are very subtle (looks like resolution differences) again questioning the value of these tests. This commit just skips the test that is failing here to unblock the PR. * Add azure 3.8 test jobs * Pin to pre-release mpl for python 3.8 * Add spawn guard for python3.8 osx in examples When running examples with execute, transpile, or any other calls using parallel_map() the new default for spawn instead of fork requires that scripts have a __name__ == "__main__" check in them to function properly. If not calls to parallel_map fail errors around the bootstrapping phase. To avoid this in the ci jobs this commit adds the necessary checks to the example scripts we run in ci as part of test_examples. * Remove cryptography pin from constraints file We pinned the cryptography package version back during the cryptography 2.6 release which broke our ci. It was a temporary step to avoid a packaging issue in CI that was blocking development. However, when the issue was resolved we never circled back to fix the issue. Now trying to enable python 3.8 support to terra cryptography >=2.8 is needed for python 3.8 on windows (to get a precompiled binary). This commit removes the unecessary pin to unblock windows 3.8 ci. * Bump cibuildwheel version to build 3.8 wheels cibuildwheel 1.0.0 was released in early November [1] and added support for building python 3.8 wheels. This commit bumps the cibuildwheel version we use in the wheel build jobs at release time to also build 3.8 wheels for upload to pypi. * Add skip and release note about macos py38 issues This commit adds a skip for the failing python 3.8 test on osx and windows so that we don't block everything over a small issue in the tests. It also add a release note documenting the limitation with python 3.8 on macos with regardess to parallel_map/multiprocessing. Since this limitation is new for this release (being the first release with python 3.8 support) we should document it as a known issue in the release notes, especially since it likely won't be resolved until a python 3.8.1 release. * Revert "Add spawn guard for python3.8 osx in examples" While we can fix the tests to work on osx python3.8 by adjusting the example scripts to only call functions using parallel_map from inside a block run via if __name__ == '__main__': this unecessarily changes the scripts for the quirks of a single environment. This commit reverts the example scripts back to their original form and instead just skips the unittest that executes them on python 3.8 macOS. We already have documented this limitation in the release notes. When/if we have an alternative solution for how we launch additional processes in python 3.8 on macOS that does not require this workaround we can look at removing the skip. This reverts commit 76ae197631247f5b18eb322b1482da9b31d5391b. * Apply suggestions from code review Co-Authored-By: Kevin Krsulich <kevin@krsulich.net>
2019-12-11 04:43:58 +08:00
before_script:
- |
if [ ! "$(ls -A .stestr)" ]; then
rm -rf .stestr
fi
Revise travis configuration, using cmake * Revise the travis configuration for using `cmake` for the several targets, and use "stages" instead of parallel jobs: * define three stages that are executed if the previous one suceeds: 1. "linter and pure python test": executes the linter and a test without compiling the binaries, with the idea of providing quick feedback for PRs. 2. "test": launch the test, including the compilation of binaries, under GNU/Linux Python 3.6 and 3.6; and osx Python 3.6. 3. "deploy doc and pypi": for the stable branch, deploy the docs to the landing page, and when using a specific commit message, build the GNU/Linux and osx wheels, uploading them to test.pypi. * use yaml anchors and definitions to avoid repeating code (and working around travis limitations). * Modify the `cmake``configuration to accomodate the stages flow: * allow conditional creation of compilation and QA targets, mainly for saving some time in some jobs. * move the tests to `cmake/tests.cmake`. * Update the tests: * add a `requires_qe_access` decorator that retrieves QE_TOKEN and QE_URL and appends them to the parameters in an unified manner. * add an environment variable `SKIP_ONLINE_TESTS` that allows to skip the tests that need network access. * replace `TRAVIS_FORK_PULL_REQUEST` with the previous two mechanisms, adding support for AppVeyor as well. * fix a problem with matplotlib under osx headless, effectively skipping `test_visualization.py` during the travis osx jobs. * Move Sphinx to `requirements-dev.txt`.
2018-02-13 05:11:28 +08:00
stage_linux: &stage_linux
<<: *stage_generic
os: linux
dist: bionic
Revise travis configuration, using cmake * Revise the travis configuration for using `cmake` for the several targets, and use "stages" instead of parallel jobs: * define three stages that are executed if the previous one suceeds: 1. "linter and pure python test": executes the linter and a test without compiling the binaries, with the idea of providing quick feedback for PRs. 2. "test": launch the test, including the compilation of binaries, under GNU/Linux Python 3.6 and 3.6; and osx Python 3.6. 3. "deploy doc and pypi": for the stable branch, deploy the docs to the landing page, and when using a specific commit message, build the GNU/Linux and osx wheels, uploading them to test.pypi. * use yaml anchors and definitions to avoid repeating code (and working around travis limitations). * Modify the `cmake``configuration to accomodate the stages flow: * allow conditional creation of compilation and QA targets, mainly for saving some time in some jobs. * move the tests to `cmake/tests.cmake`. * Update the tests: * add a `requires_qe_access` decorator that retrieves QE_TOKEN and QE_URL and appends them to the parameters in an unified manner. * add an environment variable `SKIP_ONLINE_TESTS` that allows to skip the tests that need network access. * replace `TRAVIS_FORK_PULL_REQUEST` with the previous two mechanisms, adding support for AppVeyor as well. * fix a problem with matplotlib under osx headless, effectively skipping `test_visualization.py` during the travis osx jobs. * Move Sphinx to `requirements-dev.txt`.
2018-02-13 05:11:28 +08:00
language: python
python: 3.7
Revise travis configuration, using cmake * Revise the travis configuration for using `cmake` for the several targets, and use "stages" instead of parallel jobs: * define three stages that are executed if the previous one suceeds: 1. "linter and pure python test": executes the linter and a test without compiling the binaries, with the idea of providing quick feedback for PRs. 2. "test": launch the test, including the compilation of binaries, under GNU/Linux Python 3.6 and 3.6; and osx Python 3.6. 3. "deploy doc and pypi": for the stable branch, deploy the docs to the landing page, and when using a specific commit message, build the GNU/Linux and osx wheels, uploading them to test.pypi. * use yaml anchors and definitions to avoid repeating code (and working around travis limitations). * Modify the `cmake``configuration to accomodate the stages flow: * allow conditional creation of compilation and QA targets, mainly for saving some time in some jobs. * move the tests to `cmake/tests.cmake`. * Update the tests: * add a `requires_qe_access` decorator that retrieves QE_TOKEN and QE_URL and appends them to the parameters in an unified manner. * add an environment variable `SKIP_ONLINE_TESTS` that allows to skip the tests that need network access. * replace `TRAVIS_FORK_PULL_REQUEST` with the previous two mechanisms, adding support for AppVeyor as well. * fix a problem with matplotlib under osx headless, effectively skipping `test_visualization.py` during the travis osx jobs. * Move Sphinx to `requirements-dev.txt`.
2018-02-13 05:11:28 +08:00
jobs:
include:
- name: Python 3.6 Tests and Coverage Linux
Revise travis configuration, using cmake * Revise the travis configuration for using `cmake` for the several targets, and use "stages" instead of parallel jobs: * define three stages that are executed if the previous one suceeds: 1. "linter and pure python test": executes the linter and a test without compiling the binaries, with the idea of providing quick feedback for PRs. 2. "test": launch the test, including the compilation of binaries, under GNU/Linux Python 3.6 and 3.6; and osx Python 3.6. 3. "deploy doc and pypi": for the stable branch, deploy the docs to the landing page, and when using a specific commit message, build the GNU/Linux and osx wheels, uploading them to test.pypi. * use yaml anchors and definitions to avoid repeating code (and working around travis limitations). * Modify the `cmake``configuration to accomodate the stages flow: * allow conditional creation of compilation and QA targets, mainly for saving some time in some jobs. * move the tests to `cmake/tests.cmake`. * Update the tests: * add a `requires_qe_access` decorator that retrieves QE_TOKEN and QE_URL and appends them to the parameters in an unified manner. * add an environment variable `SKIP_ONLINE_TESTS` that allows to skip the tests that need network access. * replace `TRAVIS_FORK_PULL_REQUEST` with the previous two mechanisms, adding support for AppVeyor as well. * fix a problem with matplotlib under osx headless, effectively skipping `test_visualization.py` during the travis osx jobs. * Move Sphinx to `requirements-dev.txt`.
2018-02-13 05:11:28 +08:00
<<: *stage_linux
python: 3.6
env:
- PYTHON="coverage run --source qiskit --parallel-mode"
after_success:
- coverage combine || true
- coveralls || true
- coverage xml || true
- pip install diff-cover || true
- diff-cover --compare-branch master coverage.xml || true
Revise travis configuration, using cmake * Revise the travis configuration for using `cmake` for the several targets, and use "stages" instead of parallel jobs: * define three stages that are executed if the previous one suceeds: 1. "linter and pure python test": executes the linter and a test without compiling the binaries, with the idea of providing quick feedback for PRs. 2. "test": launch the test, including the compilation of binaries, under GNU/Linux Python 3.6 and 3.6; and osx Python 3.6. 3. "deploy doc and pypi": for the stable branch, deploy the docs to the landing page, and when using a specific commit message, build the GNU/Linux and osx wheels, uploading them to test.pypi. * use yaml anchors and definitions to avoid repeating code (and working around travis limitations). * Modify the `cmake``configuration to accomodate the stages flow: * allow conditional creation of compilation and QA targets, mainly for saving some time in some jobs. * move the tests to `cmake/tests.cmake`. * Update the tests: * add a `requires_qe_access` decorator that retrieves QE_TOKEN and QE_URL and appends them to the parameters in an unified manner. * add an environment variable `SKIP_ONLINE_TESTS` that allows to skip the tests that need network access. * replace `TRAVIS_FORK_PULL_REQUEST` with the previous two mechanisms, adding support for AppVeyor as well. * fix a problem with matplotlib under osx headless, effectively skipping `test_visualization.py` during the travis osx jobs. * Move Sphinx to `requirements-dev.txt`.
2018-02-13 05:11:28 +08:00
Add python 3.8 support and deprecate python 3.5 (#3268) * Add python 3.8 support and deprecate python 3.5 This commit adds support for running terra under python 3.8. It does this by adding the trove classifiers to the package metadata and adding a test job for python 3.8 (only travis for now this can be expanded in the future when azure pipelines adds it to their image). At the same time this commit starts the deprecation window for python 3.5. Python 3.5 goes end of life by upstream python in Sept. 2020. We should give our users running with python 3.5 (which is about 10% of our users based on pypi data) fair notice that when upstream python stops supporting it we do as well. * Fix lint * Add back empty cache detection As part of the refactors the empty stestr timing cache removal code was removed from the travis config, but the cache wasn't removed. Adding the python 3.8 job causes a failure because travis has no cached timing data but still creates the empty directory which triggers mtreinish/stestr#266. This adds back the empty cache removal to workaround it so we can use the timing data for scheduling in future runs. * Add azure-pipelines python 3.8 jobs too * Fix python 3.8 dictionary keys changed error Starting in python 3.8 a new RuntimeError is raised, RuntimeError: dictionary keys changed during iteration. This is caused by modifying an iterator while looping over it. We were doing that in the optimize_swap_before_measure pass. This commit fixes this by wrapping the iterator in list() to make a copy of it for looping. This way we don't modify the contents of what we're iterating over in the pass. * Revert "Add azure-pipelines python 3.8 jobs too" The missing matplotlib wheels are blockers for windows and osx environment. Since we do not have the necessary dependencies installed in those ci envs to compile matplotlib from source. We'll rely on just travis for 3.8 testing until the matplotlib 3.2.0 release is pushed so we don't have to compile it. This reverts commit 40157621b135a03b291b93e5d9b801237a14c015. * Add skip for failing matplotlib test The image comparison tests which are quite flaky are failing with matplotlib compiled on 3.8. Looking at the image output from these failures are very subtle (looks like resolution differences) again questioning the value of these tests. This commit just skips the test that is failing here to unblock the PR. * Add azure 3.8 test jobs * Pin to pre-release mpl for python 3.8 * Add spawn guard for python3.8 osx in examples When running examples with execute, transpile, or any other calls using parallel_map() the new default for spawn instead of fork requires that scripts have a __name__ == "__main__" check in them to function properly. If not calls to parallel_map fail errors around the bootstrapping phase. To avoid this in the ci jobs this commit adds the necessary checks to the example scripts we run in ci as part of test_examples. * Remove cryptography pin from constraints file We pinned the cryptography package version back during the cryptography 2.6 release which broke our ci. It was a temporary step to avoid a packaging issue in CI that was blocking development. However, when the issue was resolved we never circled back to fix the issue. Now trying to enable python 3.8 support to terra cryptography >=2.8 is needed for python 3.8 on windows (to get a precompiled binary). This commit removes the unecessary pin to unblock windows 3.8 ci. * Bump cibuildwheel version to build 3.8 wheels cibuildwheel 1.0.0 was released in early November [1] and added support for building python 3.8 wheels. This commit bumps the cibuildwheel version we use in the wheel build jobs at release time to also build 3.8 wheels for upload to pypi. * Add skip and release note about macos py38 issues This commit adds a skip for the failing python 3.8 test on osx and windows so that we don't block everything over a small issue in the tests. It also add a release note documenting the limitation with python 3.8 on macos with regardess to parallel_map/multiprocessing. Since this limitation is new for this release (being the first release with python 3.8 support) we should document it as a known issue in the release notes, especially since it likely won't be resolved until a python 3.8.1 release. * Revert "Add spawn guard for python3.8 osx in examples" While we can fix the tests to work on osx python3.8 by adjusting the example scripts to only call functions using parallel_map from inside a block run via if __name__ == '__main__': this unecessarily changes the scripts for the quirks of a single environment. This commit reverts the example scripts back to their original form and instead just skips the unittest that executes them on python 3.8 macOS. We already have documented this limitation in the release notes. When/if we have an alternative solution for how we launch additional processes in python 3.8 on macOS that does not require this workaround we can look at removing the skip. This reverts commit 76ae197631247f5b18eb322b1482da9b31d5391b. * Apply suggestions from code review Co-Authored-By: Kevin Krsulich <kevin@krsulich.net>
2019-12-11 04:43:58 +08:00
- name: Python 3.8 Tests
<<: *stage_linux
python: 3.8
# Randomized testing
- name: Randomized tests
<<: *stage_linux
cache:
pip: true
directories:
- .hypothesis
script:
- pip install -U pip
- python setup.py build_ext --inplace
- pip install "qiskit-aer"
- make test_randomized
- name: Benchmarks
<<: *stage_linux
python: 3.5
script:
- python setup.py build_ext --inplace
- pip install "qiskit-aer"
- pip install "asv"
- git clone "https://www.github.com/Qiskit/qiskit" meta-qiskit
- >-
sed -i
-e 's/"repo": "https:\/\/github.com\/Qiskit\/qiskit-terra.git",/"repo": "..",/'
meta-qiskit/asv.conf.json
- cat meta-qiskit/asv.conf.json
- echo $TRAVIS_BRANCH
- echo $TRAVIS_PULL_REQUEST_BRANCH
- asv --config meta-qiskit/asv.conf.json machine --machine travis-ci
Change benchmarks run in CI job (#3388) * Change benchmarks run in CI job This commit changes up the benchmarks we run in the travis CI job. This job is very susceptible to noise because of shifts in CI node performance. The environment that travis uses to host their environment has variable performance being that they run on public clouds so to minimize spurious failures caused by things outside of our control we should try to limit the benchmarks we run to smaller synthetic ones then larger full path tests. To this end this commit removes the state tomography benchmark and replaces it with synthetic benchmarks for circuit construction, converters, and the assembler. The state tomography benchmarks involves running a simulation with BasicAer, ignis, and quantum_info to calculate the state fidelity and while this is a useful benchmark in dedicated benchmarking environment there is too much work going on to provide repeatable results for the CI job. * Remove converters from list of ci benchmarks This commit removes the converters benchmarks from the list while each benchmark iteration takes at most a few seconds the matrix of benchmarks run is quite large and the setup can be expensive for the larger circuits. While it only takes a few extra minutes locally on a reasonable system, in the slow ci nodes this pushes things beyond the 45min timeout. To make sure we have enough time budget this commit removes the converter benchmarks from the list we run in ci. * Remove QFT from list of benchmarks too The qft benchmarks also run for a long time and suffer from the same kind of variability that state_tomography did (although to a lesser extent). This commit removes it from the list too. * Add converter benchmarks for depth 8 Running the full range of benchmarks for the converter functions across takes too much time (mostly in setup). But they can provide valuable insights. This commit adds just the depth 8 versions of the benchmarks to the list we run in CI. This should be a tradeoff of actually running them but having it execute quickly enough in our ci env.
2019-11-08 07:25:33 +08:00
- travis_wait 45 asv --config meta-qiskit/asv.conf.json continuous --interleave-processes --machine travis-ci --no-only-changed --python 3.5 --bench 'converters.*(1|2|5|8|14|20|53)\,.8\)' --bench '^(transpiler_benchmarks|assembler|circuit_construction|isometry|import)\.' $([ "$TRAVIS_BRANCH" == "master" ] || echo "$TRAVIS_BRANCH") HEAD
matrix:
fast_finish: true
allow_failures:
- name: Randomized tests
- name: Benchmarks