* Change benchmarks run in CI job
This commit changes up the benchmarks we run in the travis CI job. This
job is very susceptible to noise because of shifts in CI node
performance. The environment that travis uses to host their environment
has variable performance being that they run on public clouds so to
minimize spurious failures caused by things outside of our control we
should try to limit the benchmarks we run to smaller synthetic ones then
larger full path tests. To this end this commit removes the state
tomography benchmark and replaces it with synthetic benchmarks for
circuit construction, converters, and the assembler. The state
tomography benchmarks involves running a simulation with BasicAer,
ignis, and quantum_info to calculate the state fidelity and while this
is a useful benchmark in dedicated benchmarking environment there is too
much work going on to provide repeatable results for the CI job.
* Remove converters from list of ci benchmarks
This commit removes the converters benchmarks from the list while each
benchmark iteration takes at most a few seconds the matrix of benchmarks
run is quite large and the setup can be expensive for the larger
circuits. While it only takes a few extra minutes locally on a
reasonable system, in the slow ci nodes this pushes things beyond the
45min timeout. To make sure we have enough time budget this commit
removes the converter benchmarks from the list we run in ci.
* Remove QFT from list of benchmarks too
The qft benchmarks also run for a long time and suffer from the same
kind of variability that state_tomography did (although to a lesser
extent). This commit removes it from the list too.
* Add converter benchmarks for depth 8
Running the full range of benchmarks for the converter functions across
takes too much time (mostly in setup). But they can provide valuable
insights. This commit adds just the depth 8 versions of the benchmarks
to the list we run in CI. This should be a tradeoff of actually running
them but having it execute quickly enough in our ci env.
In #3143 we added back most of the jobs which we removed from travis to
see if we preferred running jobs in travis or azure pipelines. After
having this for a full day it's become apparent that even with the extra
capacity from our travis subscription that the wait times on travis with
the full job load are too high. The travis check returned significantly
slower than the equivalent job in azure pipelines. So for now we'll
stick to running the majority of and required testing in azure pipelines
and use travis only for extra jobs where needed. This commit removes all
the duplicated jobs that we run in azure from the travis config.
#3143 restored travis jobs for environments other than 3.6, but
the benchmarks are currently set to run on 3.5, which needed
to be overridden explicitly.
* Add back travis config for unit tests
Since we now have a subscription for travis we no longer have to worry
as much about longer queue delays. The core team preference is to use
travis for more than for azure. This commit moves the majority of the
testing out of azure and into travis accordingly.
* Add missing echo and cache stestr dir
This commit adds a missing echo for the python hashseed so we can
actually see what we set the hash seed to. It also adds caching for the
.stestr dir which will enable seeding the test runner's scheduler with
data from previous runs to optimize the worker balance and hopefully
improve run time slightly.
* Fix typo
* Also cache stestr on osx
* Restore azure config
* Remove travis wheel build configs
* Default to python 3.7
With the recent release of networkx 2.4 several deprecated APIs were
removed. This broke the visualization dependency nxpd. A fix is in
progress [1] to address this. But in the meantime to get tests working
again we need to make sure we do not install networkx 2.4 for testing.
This commit does this by pinning the version we use in our testing in
the constraints file to 2.3. This will unblock our development while we
wait for a new nxpd release.
[1] https://github.com/chebee7i/nxpd/pull/15
The new aer packages use manylinux2010, and the randomized testing job
is using the sdist instead of the wheel. This commit updates the pip
version prior to installing aer to try and get it to install the binary
version instead of the source package.
This commit adds the import benchmark to the travis benchmark job. The
import benchmark simply times how long it takes a fresh python
interpreter to run 'import qiskit'. This benchmark executes very quickly
so it should not add significant time to the benchmark. But it will
provide useful information about the impact of our changes under
development.
* Build wheels for all platforms in azure pipelines
Azure pipelines offers hosts on Linux, macOS, and Windows so just like
for our CI test jobs we can leverage it for building all our binaries
for pypi upload. Right now we use a combination of azure, travis, and
appveyor (as a fallback for azure) to build the binary wheels and upload
them to pypi. This commit simplifies all of that and configures things
so that we only need to run jobs on a single CI system.
* Add sdist generation
Now that we moved the majority of the testing from travis into azure
pipelines we don't need to run duplicate jobs in the environment. This
will mean we return results faster for the testing we haven't migrated
yet, and also mean we use less resources to be considerate to other
qiskit projects which still use travis for the majority of their CI
needs.
This commit adds the isometry benchmarks to the set of benchmarks we run
in CI. This shouldn't significant additional time to the job run, but
will add a useful benchmarks for tracking the performance of a patch.
Most interestingly it will add a benchmark that doesn't measure time but
instead compares the CNOT count output from the transpiler. Which makes
a lot of sense in this CI job because it won't be bound to the
variable performance of the cloud guests that travis runs it's jobs on.
As part of streamlining the release process, this commit adds automation
for publishing an sdist to pypi. This was one of the last manual steps
that was part of the release process. Now there are no manual steps left
after pushing a git tag. There are still some optimizations that need to
be done prior to pushing a tag, but that is more of a process problem
rather than something that can directly be fixed by automation.
* Trial job for asv continuous in travis.
* Print circuit QASM during randomized testing.
* Add coverage to python 3.6 job name.
* Suppress hypothesis filter_too_much health check.
* Reducuce random circuit count from 50 to 25.
* Set python version via command line arg rather than sed.
* Increase travis_wait time to 45m for randomized tests.
* Bump hypothesis version.
* Decrease to 50 random circuits and fail on first first found error.
* Disable hypothesis deadlines.
* Add randomized testing stage to travis.
* Update report_ci_failure to create an issue for failed random tests.
* Remove randomized tests from unit test and coverage.
* Add hypothesis example database to travis cache.
* Add diff-cover output when coverage is collected
This commit adds an additional output stage to the python 3.6 job, which
is used for collecting coverage, that runs the diff-cover tool to show
which parts of the diff have test coverage. This is useful for both
reviewers and the commit author to see if the changes being made in a PR
are actually being exercised by the testing we run.
* Fix typo in coverage.xml filename
* Add coverage collection to CI
This commit adds coverage collection to the CI environment. It adds
reporting back to coveralls for the python 3.6 job. So when that job
finishes executing it will send the collected coverage data to coveralls
so we can track out unit test coverage.
Fixes#771
* Add missing quotes
* Move coveralls to only run on coverage job
The current wheel build automation uploads the wheel files to
test.pypi.org. The idea behind this was to enable manual verification of
the built wheels prior to pushing the releases for real. However, this
just adds another manual process to the release process and increases
the chance of something going wrong. There also isn't any manual
validation that happens for most of the wheels. When it is done, it
provides no more than what the build automation does already
(running an example script). To simplify the release process and reduce
the chances for errors involved with downloading and re-uploading
this commit switches the build automation to directly upload to
pypi.org. The release workflow with this is now just push a tag for the
release, then build and upload a sdist to pypi. Everything else is
automated.
* Install twine in wheel build jobs
The wheel build jobs failed to install twine so when the jobs completed
and went to upload the wheels this failed because the command wasn't
found. This commit fixes the issue by installing twine as part of the
wheel build job set ups so that we can actually upload our built
binaries.
* Pip install -U twine
* Bump cibuildwheel version
Now that terra and other qiskit elements have been released we can
tweak how we install packages in CI for running tests. This commit
updates things so we install terra from source, no longer pin the
ibmq-provider, and removes aer (since there are no tests using it
anymore).
* Add example for stochastic swap and use to verify wheels
This commit adds a new example to the examples directory for running the
stochastic swap pass. This is then used by the wheel build automation
to verify the built wheel works as expected. The compiled code (and
complexity in the wheel build) is for the stochastic swap pass so
verifying it functions when installed from the built wheel is important.
* Test wheel build with new script
Comment out branch logic and wheel upload to verify the wheel build this
is just for testing and will be reverted before the PR merges.
* Remove prints from example
* Revert "Test wheel build with new script"
This reverts commit f5edc1c777.
* Build wheels automatically in CI environment
Right now when it's time to push a relase of qiskit-terra we have to
manually build wheels for all our supported environments because we
included compiled code in the project. This involves setting up a proper
environment with portable librariers and compilers in Linux, Windows,
and Mac OSX with all 3 versions of python we support. Then manually
building the 9 wheels necessary to publish on pypi. However we install
qiskit-terra from source every day multiple times in all these
environments in our CI. There is no reason this manual step needs to
exist and we can just leverage CI to do our wheel building when it's
time to do a release.
This commit adds initial support for building our wheels in travis
(for linux and osx) and appveyor (for windows). It leverages the
cibuildwheel project to automate the wheel building. This will ensure we
setup the proper environment for all 3 python versions on all 3
environments (for example pulling in the manylinux docker container on
linux).
* Add cython to wheel build environment
* Add sudo call to pip
* Skip python 3.4 wheel build
* Fix appveyor syntax
* Setup seperate wheel job on appveyor
* Fix missing quote
* Fix osx env skip typo
* Add wheel tests and twine config to CIBW jobs
This commit adds 2 things. First running tests on the built wheels. This
will enable verifying that the built wheels work by running the unit
tests with terra installed from the wheels. The second is the beginnings
of the twine config. In the next iteration of this patch the wheels will
all be uploaded to test.pypi.org so we can test them locally and then
uploaded them to pypi.org for release on each tag push. (We might bypass
this and go straight to pypi.org in the future) To facilitate this we
need to pass configuration into twine so for when we add twine upload to
the builds it will be able to upload.
* Add missing quotes to test command
* Run tests outside of project directory
* Switch CIBW tests to use unittest instead of stestr
Running tests with stestr assumes you're running inside the python
project for discovery to work. Since we don't want to do that because it
will get confused about whether we mean to run with the installed wheel
or local files. This commit switches to use unittest which will enable
running discovery from any directory so we can run tests against just
the installed wheel to verify that it works correctly.
* Run examples script instead of unittests
The unittests are too coupled to running from inside the repo. Since we
just need to verify the installed wheel works this switches to running
one of the example scripts, teleport.py. This is a good choice because
it will use both python code and cython compiled code to verify the
wheel works as expected.
* Remove unused test requirements
* Fix windows backwards slashes
* Update windows test command
* Add twine upload and only run on tags
* Run wheel build as a deploy on tag in appveyor
* Enable deploy on each run for testing
* Pivot appveyor config to use conditional matrix instead of deploy
* Add twine upload to appveyor
Right now the jobs listed in travis don't have explicit names. You can
pretty easily figure out which jobs do what based on the OS icon and the
environment variables set for each job. But to make it a bit easier just
name each job with a descriptive human readable name so we're explicit
about which each job is doing.
* Add constraints to pin pylint to known working version
This commit adds a constraints file so we're always running pylint and
astroid at known working version. The 2 are requirements are very coupled
and small changes in one often requires an update in the other.
Unfortunately new versions of pylint often change or add rules which
makes running it unpinned not a good decision. To ensure a consistent
environment for running pylint this commit leverages pip constraints to
make sure we always install the same version of astroid and pylint.
* Pin for broken cryptography release today too
* Add Cython files.
* by error
* Modify swap mapper
* cleanup setup.py
* move cythonize to main setup.py
* fix style and add cython to requirements
* have travis build cython before testing
* disable lint checks that fail due to compiled code
* add cython to requirements (the txt one)
* revert to NumPy RandomState RNG for deterministic random generation
* remove unused import
* update tests
* make Cython build more portable
* move cython to dev requirements
* remove unused code
* turn on warnings, and fix old docstring
* fix import errors
* add cython build to appveyor
* add cython to appveyor
* revert aer changes
* fix "ground truth"
* this should not be here
* this should not be here
* this should not be here
* this should not be here
* this should not be here
* this should not be here
* I have no idea why they are saying they are modified
* Add Cython to setup_requires for setup.py
The setup_requires parameter to the setup() function is used to
specify packages that need to be present in order for the setup script
to run. [1] With the introduction of Cython to the setup.py this is now
required to be installed for setup, so this commit adds Cython to
setup_requires to indicate this.
[1] https://setuptools.readthedocs.io/en/latest/setuptools.html
* add to changelog
* Update Cython version in setup_requires
* Aggressively caching pyenv directory.
Most of the time spent on iOS instances came from:
* Updating brew so pyenv is able to install the latest Python
versions.
* Building the Python versions.
The patch removes the need of brew by installing pyenv directly. This
pyenv is then used to build the three Python interpreters that we
support. Now, we cache these builds. Finally we setup the Python
environment in the same way we did before the patch.
Now the fixing downloads and build the three environments thrice in the
worst case (when there is no cache) and updating a Python version, (which
is expected for Python 7) requires clearing the cache.
* Now each version is responsible of the proper Python version
* Restoring tests
* Utility for reporting a CI failure as a GitHub issue.
This PR adds a new tool for creating an issue on GitHub when the
CI fails. The utility identifies the report with the name of the
branch and commit hash an avoids to open two reports for the same
subject.
* Fixing the master is failing label
* Add automatization in CI
* Add test_ci make target, revert make test
Add a new "test_ci" target to the Makefile, aimed at being used by
the CIs. Update the "test" family of targets in order to revert
back to the behaviour that involves less overhead and complexity
for the users.
* Update travis in order to use test_ci
* Add documentation about test_ci target
* Remove documentation deploy
Remove documentation script and travis job.
* Remove deploy stage from travis
Remove the "deploy doc and pypi" stage entirely, and revise the
"subclassing" of the osx jobs.
* Fix travis script make invocation
* Fix extra lint script commands
* Pin numpy version to <1.16
Pin the numpy version to <1.16 until the potential incompatibilities
are fixed.
* Remove numpy from whitelist, ignore random instead
Remove `numpy` from the `extension-pkg-whitelist`, and instead manually
ignore the check for the members of `numpy.random`, as it was the
only place where the check was significant.
Currently the osx jobs are failing trying upgrade the pyenv version to the latest (which is needed to install the python versions we test in CI). According to the travis docs the best way to use homebrew on osx jobs is to rely on the travis addon instead of doing it manually in the install scripts. This commit makes this change so we install the latest version of pyenv (and the python 3.7 dependencies) via the addon instead of out of band. This fixes the failure and makes sure we have the latest versions of the homebrew package installed as needed.
* Find working pyenv versions on osx homebrew
* Switch to travis addon for brew
* Rework travis osx config a bit more
* Remove legacy cpp code
* remove legacy cpp sim from make and travis
* Rename decorator and folder for aer provider tests
* remove Cpp simulator for setup.py
* fixup
* Remove all cmake and cleanup travis config
Since we're no longer building any C++ code there is no need to keep
cmake usage around anymore. This commit removes the last cmake remnants
and also cleans up the travis config to simply run make && make test to
install everything and then just invoke the test runner. It also
deduplicates jobs now that there is no distinction between no_compile
and compile jobs.
* Fix some small oversights
* Remove legacysimulator contributing doc section
* Removing simulators
* Fixing some test
* Fixing test
* Test
* Email
* Fixing test
* Passing locally
* Hack to get aer
* Diego fix
* Trying to fix windows errors
* Aer to basicaer
* Adding lint
* Removing test around and making aer only on one folder.
* Adding HACK FROM THE DEPTHS OF DESPAIR AS AER DOES NOT WORK ON MAC
* Oops Diego hack I did not implement correctly
* Linting
* do not import Aer at the top of tests
* lint
* two more CHANGELOG lines
* Add draw() method on QuantumCircuit class (fixes#1004)
Adds a draw() method to the QuantumCircuit class for
visualizing the circuit. This is basically just an alternative interface
to qiskit.tools.visualization._circuit_drawer() but is more convenient
because it's baked into the quantum circuit. This means users don't have
to import an additional module and can just visualize any circuits after
creating them.
* Add __str__ property to call text output draw to quantumcircuit
This commit adds a __str__ method to the quantumcircuit object. This
will enable users to run print(circuit) and have the text visualizer
output be printed from the circuit.
* Fixes final states for `get_status_job()` support
* Factor out API final states
* Moving job final states to a module level variable to keep consistency with API final states.
* Use Xenial and sudo: true to make Python 3.7 builds to work.
* Adding Python 3.7 to the Travis Matrix for Linux and OSX.
Waiting for Appveyor to support Python 3.7 natively before adding it to the matrix.
* Used pyenv and venvwrapper for both OSX stages.
* Update OSX Python 3.7 stage key
Use fixed Python version on travis osx
Fix the travis python version to 3.6.5 using `pyenv`, as the current
runs use Python 3.7 which seems to cause some issues in some of the
dependencies.
Rename all references to `qiskit-sdk-py` to the new repository name,
`QISKit/qiskit-core`, including a handful of instances where the
`IBM/qiskit-sdk-py` form was used.
* CMake improvements:
* Static linking is not the default anymore.
* Platform Wheel Tag is now configurable via cmake variable.
* Updated documentation
* CI config file updated accordingly
* Added support for Win64 builds on Visual Studio 2017
* Windows C++ Simulator now links with a prebuilt version of
OpenBLAS (64 bits), so there's no need to link against
LAPACK anymore
* cmake added as a dependency
* Restored --static-libgcc flag for static MacOS builds so gcc
can create *semi* static executables again. Clang doesn't have
static compilation.
* Remove default WHEEL_TAG value so if not specified, the tag is
choosen by the wheel module
Matthew Treinish