* make compatible with pylint and pycodestyle
* black config
* flake8 compatible config
* Missing newline
* Add black check to CI and dev requirements
This commit updates the CI configuration and local tox configuration to
leverage black instead of pycodestyle. It adds a check job to ci and the
tox lint job so we can quickly check if black has been run. A tox job
named 'black' is added to run black on all the code in the repo.
* Run black on everything
This commit reformats all the code in the qiskit-terra repository to use
black. It changes no functionality in the project and just adjusts the
code formatting to be consistent and automated. If you are looking at
this commit in the git log you can likely safely ignore any diff from
this commit as it is just the result of running '`black' on the repo
and instead you should look at any commits before or after this for
functional changes.
Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>
* Add macos universal2 wheel job
This commit adds a new wheel job for building universal2 wheels for
macOS. The universal2 wheels will enable running on arm64 macOS systems.
As we don't have any arm64 mac hardware available in CI systems yet
these jobs are cross-compiling arm64 binaries from an x86_64 mac, we
also can't test the arm64 binary component as part of the job because of
this.
This commit also takes the opportunity to update the cibuildwheel
version used for CI to the latest release 1.10.0, since we need to use
>=1.9.0 for arm on macOS support.
* DNM: Test wheel biulds
* Fix job name
* Revert "DNM: Test wheel biulds"
The wheel builds all worked and @nonhermitian manually tested the
universal2 binary on an arm64 mac. So this commit removes the test
commit to force the wheel jobs from running on the PR so this is ready
to be merged.
This reverts commit a9bafddcb8.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
With the pending support being added to qiskit-bot in
Qiskit/qiskit-bot#9 we're able to rename the default branch for the
repository to 'main'. However, when we do that several things will need
to be updated, most importantly the CI trigger was hardcoded to the
previous default branch 'master. This commit fixes these references and
should be merged after we rename the branch to re-enable CI.
* Cleanup stestr caching to only what's necessary
stestr maintains a local repository of run history which is used for
both local debugging (like for automated bisecting of failures between
runs) and also to provide data for parallel test scheduling based on the
the previous run time. In #5451 we added a cache step to the azure ci
configuration to cache the .stestr directory (the default location of
the stestr local repository) to preserve the run history to improve
parallel test scheduling to try and improve throughput. This however had
the unintended side effect of growing the stestr repository directory
significantly. Right now the stestr cache is several gigabytes in size.
This is because stestr will keep a history of all the runs in that
directory and by preserving it in a cache it's growing every time we
commit to the branch. However since we're only concerned with the timing
data we don't need the result streams for the runs to be preserved. This
commit adds a step to the CI jobs before we update the cache to remove
the results streams (which are just integer filenames in the repository
directory) and leave the dbm database of timing data. This will
significantly reduce the size of the data to cache, and also will limit
cache size growth to be a function of the number of tests in the suite
vs the number of runs in CI.
In a future stestr release there will be a command we can run to do this
for us (see mtreinish/stestr#306) but until that is released this commit
just manually deletes all the unecessary files prior to updating the
cache.
* Fix copy and paste error
* Workaround -d not existing in macOS xargs
* Workaround macOS grep not having -P
* Modify linux 3.7 CI job to install from sdist
Previously we were never explicitly testing the generation of an sdist
and whether we could build a package from that generated sdist. This
caught us by surprise in the 0.17.0 release because of an oversight
in #5941 we forgot to update the MANIFEST.in to include the new cython
source files which resulted in an sdist that could not be built. To
prevent potential issues like this in the future this commit modifies
one of the existing test jobs to explicitly generate an sdist and
install from that sdist instead of just pip installing the repo. This
will prevent such issues in the future.
* Try running in a different directory
* Make test suite self contained
When modifying a test job to build and install from a generated sdist
and run the unit tests from a non-repo directory a set of tests began to
fail because they are written with the implict assumption that tests are
always run from the git repo. This is primarily because the base test
class lives in the qiskit namespace which gets installed on install. In
the base test class there is a method used to find the path to resources
used for tests but that is written relative to the base class file which
at install time is not guaranteed to live in the same relative path to
the tests as when they're in the source repo. Since several other qiskit
projects rely on that base test class this commit just updates all the
tests which were relying on this method and have them directly use a
relative path from the test file so that all the tests work regardless
of how you install qiskit.
* Fix junitxml reporting
* Fix schemas test paths
* Fix typo
* Add example schemas to sdist too
* Fix lint
* Update test/python/test_qasm_parser.py
Co-authored-by: Lev Bishop <18673315+levbishop@users.noreply.github.com>
Co-authored-by: Lev Bishop <18673315+levbishop@users.noreply.github.com>
* Add lightweight v2 provider interface starter
This commit is a lighterweight v2 provider interface. This is an
alternative to what is built in #4885. While the model in #4885 is a
desireable end state it requires a lot of changes all at once for
providers and potentially users. Instead this commit brings the core
concept from #4885 of a cleaner explicitly versioned abstract interface
but minimizes the changes to the data model used in v1. Only some small
changes are made, mainly that jobs can be sync or async, Backend.run()
takes a circuit or schedule, options are set via an Options object at
__init__, with set_option(), or as kwargs on run(). In all other places
the object models from the v1 provider interface are used. This makes
the migration to a versioned interface simpler to start. From there we
can continue to evolve the interface as was done in #4485, like moving
to a target object, reimplementing properties and defaults as versioned
objects, etc.
Since the differences here are so small this commit brings the basicaer
provider over to the v2 provider interface. This can be done with
minimal effort to showcase how similar the 2 interfaces are.
* Fix basicaer simulator init
* Fix lint
* Add provider property to basicaer for Aqua backwards compat
* Add provider method back to backend class for backwards compat
* Fix lint
* Add release notes
* Add v2 provider to docs
* Fix lint
* Revert basicaer v2 provider migration
* Apply suggestions from code review
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>
* Add missing version attributes
* Make Options a simplenamespace subclass
* Update Backend docstrings
* Add v2 Backend support to the rest of terra
* Fix lint
* Fix lint
* Flatten providers subpackage
* Apply suggestions from code review
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Update release notes
* Migrate basic aer provider to v2 interface
This commit migrates the basic aer provider to the v2 interface. This
was originally included in #5086 but had to be removed because of a
potential backwards compatibility issue with aqua when using basic aer
as the provider (aqua 0.7.x explicity checks for v1 interface backends).
* DNM install aqua from source to test tutorials
* Remove deprecated schema validation
* Test failures
* Fix tests and lint
* Install aqua from source until release
* Add release notes
* Add ignis from source too as a dependency of aqua
* Apply suggestions from code review
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Finish upgrade release note
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>
In #5988 we changed the character encoding used in windows CI jobs so
that it was explicitly using utf8 to avoid issues with characters being
emitted from the tests outside the default windows character encoding.
This was coming up because even though python is explicitly utf8 the
shell or the pipes used for interprocess communication in stestr did not
know we were using utf8 and would raise an encoder error when trying to
write out a string with utf8 characters. However, what that PR neglected
was the later step in the windows test jobs that we pipe the subunit
stream for the test run into a junitxml generator which would also
suffer from this problem. However we did not encounter this because only
the test id, runtime, and any failure or error messages are written out
on the process across the pipe so unless we were using characters
outside the default windows code page we wouldn't have any issues. But
as was recently discovered when a test error message included 'π' the
generate results step will fail on windows jobs. To address this edge
case this commit adds the same env variables and chcp command to the
generate results step of the windows test jobs so that if we have error
messages with utf8 characters in the output.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Set character encoding in windows CI
This commit is an attempt to fix the CI configuration around character
encoding in windows when running tests. The test jobs all rely on stestr
for running the tests which spawns multiple worker processes to execute
the tests using python's stdlib subprocess.Popen. This however can
cause issues with the default character encoding on windows (as reported
in mtreinish/stestr#295) where either the worker processes or the pipe
used for interprocess communication on windows does not carry over the
same character encoding as the parent process and reverts to the default
windows encoding of cp1252. This commit tries setting the character
encoding both through the change code page command and through setting
the LANG env variable.
For testing purposes a test is added to purposefully emit utf8
characters to stdout to verify if this works or not. This can be removed
prior to merging (although it doesn't do any harm by itself).
* DNM: Test things fail today
* Revert "DNM: Test things fail today"
This reverts commit 73c3d5f5b2.
* Re-enable logging on algorithms/opflow windows unit tests
* Fix lint
Co-authored-by: Manoel Marques <manoel.marques@ibm.com>
Co-authored-by: Luciano Bello <bel@zurich.ibm.com>
This commit reorganizes the stages and job structure of our CI jobs to
try and improve overall project throughput. It moves to a 3 stage
pipeline instead of the 2 we used before. The first stage runs python
3.7 on all OSes and lint, the second runs the remaining unittest jobs
(for 3.6, 3.7, and 3.8), and the final stage runs the docs builds and
tutorials job (which are the slowest jobs). The goal is by splitting
thing up into more stages and making the slowest jobs run last that this
will improve overall efficiency of CI.
Co-authored-by: Kevin Krsulich <kevin.krsulich@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
We're histting a relatively high tutorial job failure rate caused by
jupyter/jupyter_client#541 on certain notebooks we run in the tutorials
job. This appears to be caused by multiprocessing usage inside the
qiskit that print to stdout while running. To try and avoid this issue,
this commit disables parallelism in qiskit by setting the env var to do
that. The only concern is whether we have sufficient time budget to
execute the notebooks in CI.
* Attempt to install latest numpy in all CI jobs
Since the pulse simulator in aer 0.7.4 release appears to not be
compatible with numpy releases <1.20.0 (see Qiskit/qiskit-aer#1120) we
need to ensure that we're running jobs that use aer with numpy 1.20.0
too. This commit attempts to ensure we're using the latest numpy release
in all CI jobs to unblock CI while we wait for a fixed aer release.
* Try to pin with constraints
* Fix issues with windows and 3.6
* Update azure-pipelines.yml
Co-authored-by: Kevin Krsulich <kevin@krsulich.net>
* Fix lint error now being caught in AbelianGrouper
* Wrap qiskit.Aer and qiskit.IBMQ with lazy loading object
This commit migrates the qiskit.Aer and qiskit.IBMQ module attributes to
be lazy loading instances of lazy loading wrapper classes. The intent
here is to avoid importing from qiskit-aer or qiskit-ibmq-provider from
qiskit-terra, while this is safe while the packages share a shared
namespace we've been actively working to remove the use of namespace
packaging (see #5089, #4767, and Qiskit/qiskit#559) and the circular
dependency caused by re-exporting these attributes is blocking progress
on this. By using a lazy loading wrapper class we avoid an import type
circular dependency and opportunistically use qiskit-aer and/or
qiskit-ibmq-provider at runtime only if they're present after everything
is imported. This also may have some benefit for the overall import
performance of qiskit (being tracked in #5100) as it removes qiskit-aer
and qiskit-ibmq-provider from the import path unless they're being used.
Although the presence of qiskit.__qiskit_version__ might prevent any
performance improvements as it still imports all the elements to get
version information (and will be tackled in a separate PR).
Fixes#5532
* Fix lint
* Fix test lint
* DNM: test with ignis patch
* Revert "DNM: test with ignis patch"
This reverts commit ac9611c3ac.
* Use ignis from source for tutorial job
* Update release note to be more clear
* Add script to verify slow optional imports aren't used in default path
This commit adds a new script that is run in CI to find whether slow
optional imports are in the default qiskit import path. We've recently
had several instances of PRs being pushed that added sympy to the
default import path which makes the overall qiskit import significantly
slower (see #5576 for the most recent occurrence), this script will
catch those siutations and report the error.
* Use virtualenv in CI for running new script
The azure ci env was using system python instead of the venv where
qiskit-terra was installed for running the new script. This was causing
an error because to work the script imports qiskit to verify nothing
slow is getting imported. This commit fixes this by explicitly using the
venv python instead of the default system python to execute the script.
* Add script to makefile too
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
The stestr [1] test runner we use for CI has support for tracking
historical test results. It can then use that history to optimize the
scheduling of tests across the workers. Without any data it just splits
tests in alphabetic order which can lead to idle test runners. This is
something we're seeing in CI where one worker typically runs for several
minutes longer than the other. This commit attempts to address this
situation and optimize the scheduler by caching the stestr history dir
between runs so we can optimize the scheduling of tests.
[1] https://stestr.readthedocs.io/en/latest/index.html
Co-authored-by: Kevin Krsulich <kevin.krsulich@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Bump cibuildwheel to latest version
The linux wheels we publish to PyPI at release time are based on the
manylinux1 [1] and manylinux2010 [2] packaging specifications. To build
these wheels we use the manylinux2010 docker image [3] which is based on
centos 6. Centos 6 recently went end of life and the yum repositories
for it were deleted and an archive url for the yum repositories were
added. However, until the latest manylinux2010 docker image, the images
were configured to use the old url from when the distro was still
supported by upstream. This causes issues when trying to setup a build
[4] because yum will error early during the initial job setup. To fix
this we need to ensure we're using the latest version of the manylinux
2010 docker image, we can either do this manually (as was done in
Qiskit/retworkx#212 and Qiskit/qiskit-aer#1066) or just update the
cibuildwheel version we're using to the latest release 1.7.1 which
bumped the pinned manylinux2010 docker image used to one with updated
yum urls (see joerick/cibuildwheel#472). This commit bumps the
cibuildwheel version to the latest release so that when it is time to
release terra 0.17.0 the jobs will work as expected.
[1] https://www.python.org/dev/peps/pep-0513/
[2] https://www.python.org/dev/peps/pep-0571/
[3] https://github.com/pypa/manylinux
[4] pypa/manylinux#836
* DNM: test wheel jobs
* Revert "DNM: test wheel jobs"
This reverts commit b63071a6b3.
Co-authored-by: Kevin Krsulich <kevin.krsulich@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add support for Python 3.9
Python 3.9.0 was released on 10-05-2020, this commits marks the support
of Python 3.9 in qiskit-terra. It adds the supported python version in
the package metadata and updates the CI configuration to run test jobs
on Python 3.9 and build python 3.9 wheels.
* Also deprecate 3.6
* Use latest nodes for CI
* Remove Python 3.6 deprecation
The Python 3.6 deprecation is being handled separately in #5301 to
decouple it from adding python 3.9 support. This commit removes the 3.6
deprecation accordingly.
* Bump cibuildwheel version to the latest release
* Remove unused assertNoLogs method
The assertNoLogs methods was built using a private class from python's
stdlib unittest library. This should never have been done as it's
explicitly marked as private. Accordingly in Python 3.9 this private
class has been removed and no longer exists. It turns out this method
was not used anywhere in all of qiskit (not just terra). Since the
implementation is not sound and nothing uses it this commit just removes
the class and method.
* Fix lint
* Remove aer from macOS and windows jobs
* Remove IBMQ provider from ci jobs
The ibmq provider is not used by anything in CI anymore. It was an
historical artifact when some of the visualization and jupyter tests
required it. This is no longer true. However, installing the ibmq
provider in the linux 3.9 job is blocking CI because of an unrelated
requests library issue with urllib3. To resolve this, this commit just
removes the unecessary install.
* Add back ibmqprovider to tutorials job, it's needed for aqua tutorials
* Fix jupyter test skip
* Update test/python/tools/jupyter/test_notebooks.py
Co-authored-by: Christian Clauss <cclauss@me.com>
* Fix lint
* Add back pbars macOS test skip
Co-authored-by: Christian Clauss <cclauss@me.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
The 07_asian_barrier_spread_pricing tutorial is quite slow and often
times out in CI because running the simulation takes very close to 3
minutes in the fastest case. But, the performance of the CI nodes is
quite variable since they run on virtual machines in azure's public
cloud. So many jobs fail randomly because of timeouts on that notebook.
In Qiskit/qiskit-tutorial#1091 we added an env var to adjust the per
cell timeout for CI jobs. This commit leverages that env var in an
attempt to fix the timeout issues by bumping the timeout to 5 min which
should hopefully give us a healthy margin for slow CI nodes. Especially
since the upstream tutorials repo CI is still holding incoming PRs to 3
mins per cell.
* Pin docplex version in constraints.txt for tutorials CI job
The recent new release of docplex has broken the released version of
aqua. There is a fix in aqua in-progress Qiskit/qiskit-aqua#1420, but
until that is included in a release we won't be able to use the latest
version of docplex in the tutorials CI job. This commit adds a pin on
the version of docplex used in the constraints file to always use a
known working version to unblock CI.
* Remove windows pip cache
The windows CI images seem to have been updated and now the pip cache
step can't find the local pip cache to upload to azure. To unblock CI
this commit just removes the windows caching from CI. It will add some
execute time, but that's still better than not working.
This commit reverts a change made during #5083 to use aqua from master
for the tutorials job. This was done to workaround an aqua bug with
a non-u1/u2/u3 basis that was occuring with the 0.7.x release of aqua.
We typically use the released versions of the libraries in these jobs to
catch potentially breaking changes for the biggest users of the terra
api (the other qiskit elements). This restores this coverage by
switching back to using the aqua release for the job.
* fix first round of deprecation warnings
* fix more deprecation warnings
* add U/P to ast_to_dag
* update examples
* update circuit library
* update controlled gates and UC
* circlib
* Apply suggestions from code review
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* more deprecations
* update deprecation warnings with sx-p decomps
* update test_parameters
* update test_barrier_before_final_meas
* example in qiskit/transpiler/__init__
* update test_assembler
* test_circuit_to_instruction
* dagcircuit.test_compose
* test_dagcircuit & dependency
* qi.test_synthesis
* qi operators/states
* test_basis_translator
* test_collect_2q_blocks
* commutative analysis & cancellation
* transpiler.test_consolidate_blocks
* transpiler.test_cx_direction
* transpiler.test_decompose
* transpiler.test_kak_over_optimization
* transpiler.test_optimize_1q_gates
* test_passmanager & preset_passmanagers
* transpiler.test_remove_diagonal_gates_before_measure
* transpiler.test_stochastic_swap
* transpiler.test_unroller
* visualization tests
* randomized.test_synthesis
* scheduler.test_basic_scheduler
* pulse.test_builder
* qft examples
* transpiler/graysynth
* fix qubit broadcast in U2
* fix lint
* typo: circular->cyclic
* fix lint
* fix lint from merge
* rm deprecated call to u3
* Update qiskit/circuit/quantumcircuit.py
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Update qiskit/circuit/quantumcircuit.py
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* apply suggestions from code review
* make U a base gate
* update tests to U as base gate
* temporarily run tutorials on aqua master
* wrong linenumber in todo
* remove comment of old decomposition
* add reno
* consistent use of pi symbol in warnings
* update test to not use deprecated names
* fix messed up ff merge
* no clue where that » went
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
Co-authored-by: Kevin Krsulich <kevin.krsulich@ibm.com>
* Image comparison tests in CI
* bad tab
* valid name
* env
* set
* env
* move it
* lint
* RESULTS
* New bash
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
* Add missing __init__.py to template_circuits
This commit adds a missing __init__.py file to
qiskit/circuits/library/template_circuits, without this __init__.py the
template_circuits directory is not a valid python package. This means
it's qiskit.circuit.library.template_circuits is not a valid python
path, which means tooling that relies on this in python like unittest
discovery or critically in this case setuptools.find_packages() would
not see this package and it would be excluded. This results in
ModuleNotFoundErrors when installing the package. However, in the case
where an editable install was used this wouldn't cause a hard error
because the editable install is just a symlink to the source repo and
since the module was only accessed via relative imports it would be
found. Adding the missing __init__.py will fix this issue in the
scenarios it was broken.
At the same time to catch issues like this in the future the pip install
-e is removed from one of the CI jobs to ensure we install the package
instead of just symlinking. This will catch issues with missing packages
since we have setuptools copy the files from the package at install
time, so if we're missing an __init__.py this will fail tests since it's
not in the installed copy.
Fixes#5058
* Add copyright header to empty file to appease lint overlords
* Add support for attachments to test execution
By default in CI and with tox locally we run all the tests in parallel.
However, this makes things like printing to stdout, warnings on stderr,
or logging during tests difficult to deal with. This is because we'll
have multiple writers to the output streams simultaneously and no way
to figure out which text goes to which test. Luckily stestr and the
subunit protocl it uses internally has a method for adding attachments
to test results to handle this kind of case. Using this each output
stream is associated with the test which generated it and will get
displayed as such. The only issue with this is that the stdlib unittest
doesn't know how to write attachments to it's result stream. There is a
unittest extension library testtools[1] that provides this functionality
in a strictly compatible manner with unittest, however the current state
of the library needs some work before it can work well with more modern
features of stdlib unittest from python>3.5 so we either have to change
the tests to not use these features or come up with an alternative
solution for test attachments.
This commit ports the necessary result streaming functionality from
testtools into the QiskitTestCase base test class and then sets up the
appropriate fixtures to attach output streams from each test. This works
around the current limitation with testtools (I'm slowly working on
fixing this in testtools) but still provides the functionality. When
testtools is at the point it can be a drop in replacement for unittest
as terra is using it we can drop this code.
As part of this change all the test classes that define a setUp have a
missing super() call added. This framework does not work without setUp
in the base class being defined since that's what is used to setup the
stream capture. Moving forward the base test class actually enforces
that super() is called, otherwise the tests will fail.
[1] https://github.com/testing-cabal/testtools
* Fix lint
* Update contributing doc
* Fix copy paste error
* Add missing super calls from new tests
* Add missing super() calls and fix skip message
* Fix lint
Co-authored-by: Luciano Bello <luciano.bello@ibm.com>
* Add tutorials job to CI
This commit adds a new job to run the qiskit tutorials in CI. The
tutorials are used as a form of upgrade testing to ensure that for key
examples Qiskit as a whole is N and N+1 releases. This caused friction
during the qiskit 0.20.0 release because the tutorials were never
updated to stop using deprecated code (or in the case of other elements
by merging backwards incompatible changes). To ensure there aren't any
surprises at the last minute when we run the tutorials with a proposed
new metapackage this commit adds a job to CI to ensure that they always
run with terra changes. It means for PRs that change an api (after a
deprecation cycle) the tutorial will need to be updated first. This will
ensure that users will have an upgrade path because CI in
qiskit/qiskit-tutorials runs with the metapackage.
* Install pandoc in tutorials job
* Remove aqua tutorials from ci job for run time
* Fix rm path and speed up tutorials clone
* Try pinning matplotlib
* Try fixing archive path
* Add back aqua tutorials
Co-authored-by: Luciano Bello <luciano.bello@ibm.com>
Co-authored-by: Kevin Krsulich <kevin.krsulich@ibm.com>
This commit drops support for running with python 3.5. It marks the
minimum supported version of the package as python 3.6, removes python
3.5 package pins, removes the 3.5 CI jobs, and removes the warning on
python 3.5. Looking at the PyPI stats since the deprecation period
started the number of users on Python 3.5 has diminished significantly,
but not disappeared. There were 783 downloads with pip from pypi out of
total of 25782 total pip downloads in the last 30 days. Compared to the
roughly 10% figure when we deprecated Python 3.5.
Merging this means we can not release until after the documented EoL
date for Python 3.5 support of September 13. This shouldn't be a problem
because with #4767 we will need to coordinate the release of all the
qiskit elements and are planning to do that after 09/13/2020.
It's worth noting that we should start planning to deprecate python 3.6
support sooner rather than later it goes EoL upstream at the end of
next year [1] and some of our other upstream dependencies (mainly numpy
et al) are going to remove support before the upstream Python EoL date
[2].
[1] https://devguide.python.org/#branchstatus
[2] https://numpy.org/neps/nep-0029-deprecation_policy.html
Co-authored-by: Kevin Krsulich <kevin.krsulich@ibm.com>
* Isolate ci test environment with virtualenvs
Right now CI jobs are failing because of the dep solver because of a
version conflict between 'aws-sam-cli' and the dateutil version terra
requires. We're not installing aws-sam-cli as it's not in the dependency
tree for terra, so presumably it's coming from the system python
environment. This commit uses virtualenv to create an isolated
environment for running tests to avoid these types of interactions in
the future.
* Windows is different
* Add tools/verify_headers.py to check copyright headers
Currently we're relying on a pylint plugin to verify copyright headers
conform to the project guidelines. However this is problematic for a
number of reasons. The first is it has proven to be unstable there have
been several instances of both users and CI failing on header files that
do not have any issues. This is pretty common with pylint in general
since it's very sensitive to differences in environment. Additionally,
it was using a regex to try and match the entire header string. This
provided no practical debugability when it fails (especially when the
failure is unexpected) because we end up with only a match error.
This commit addresses these issues by removing the use of the regex
based pylint plugin and adding a small script to verify the header in
every qiskit file and provide useful debugging if there is an issue with
a header in a python file.
Fixes#3127
* Add toxinidir to tox script call
* Explicitly set character encoding for windows
In manual testing running the verify_headers.py script on windows would
fail because it was trying to use the system default character encoding
instead of utf8 and would encounter chartacters outside the windows
charmap encoding. To avoid this issue this commit explicitly sets the
character encoding to utf8. We already declare that in the header (which
is part of what this script is verifying) so we can assume all files are
utf8 encoded and if they're not the script will catch it.
Co-authored-by: Luciano Bello <luciano.bello@ibm.com>
Azure pipelines caching task does not update the cached data for a given
key after it's been stored. This means our pip cache is never being
updated. So the effects of #4346 can't be seen because the built wheels
are not actually being cached between runs. Also, when new versions of
requirements are released the cache will not be updated with them
either which will eventually mean we have no cache hits. This commit
updates the key string used for the cache to include the build number
meaning we will always upload the contents of the pip cache after each
run. This will enable storing new entries in the cache so we keep it up
to date.
This commit updates the ci configuration to ensure that we have wheel
(and setuptools) installed and at the latest version. Right now in the
azure jobs we cache dependencies but it's not caching locally
constructed wheels from installation. This means the effect of the cache
is minimized to just the wire time to download artifacts from pypi, To
improve the effect of the cache this installs the necessary requirements
to enable pip to build wheels locally when an sdist package is used.
When that happens it will start caching the wheels and saving us from
building from source on every run.
Co-authored-by: Julien Gacon <jules.gacon@googlemail.com>
* Add missing succeeded() to stage conditional
As follow up to #4343 and the third attempt to get auto cancellation to
work fully this commit adds a missing succeeded() to the lint and test
stage conditional. Without this the stage is still getting triggered
despite the auto cancellation. This should hopefully be the last attempt
as currently the second stage is getting cancelled (at least some of the
time) which means the config is partially working.
* Remove the stage conditional
We can rely on the per job conditionals to skip on tags (so we only
build wheels and the logic is hard to get right between the heiarchy of
conditions.
* Remove all job conditions and rely on stage behavior
This commit removes the conditions to skip running tests on the tags.
These were in place to avoid running tests twice when we push a release,
but having the conditions complicates the skip logic too much when
coupled with the auto-cancellation rules. This commit just removes the
conditions so we get PR CI throughput back, at the cost of running
duplicate tests on tags.
* Fix dependency between stages
* Modify job conditions to enable auto cancellation
As a followup to #4342 the auto cancellation is still not working.
Searching for this issue yielded scipy/scipy#11295 and it's fix
scipy/scipy#11301. This commit duplicates that for the terra config in
the hopes of making autocancellation work again.
* Add missing close parenthesis
Co-authored-by: Kevin Krsulich <kevin.krsulich@ibm.com>
Currently the azure pipelines jobs are not being cancelled for PR
updates before a job finishes. According to the docs [1] this is
supposed to be default 'true' however we have had issues with implicit
defaults in the past on azure. This commit makes the configuration
explicit in the hope that it will re-enable the feature and improve
CI throughput.
[1] https://docs.microsoft.com/en-us/azure/devops/pipelines/repos/github?view=azure-devops&tabs=yaml#multiple-pr-updates
* Add pip caching to azure pipelines jobs
Previously azure pipelines did not have fully supported caching support
but that has since changed. This commit adds configuration to our azure
pipelines jobs to leverage a pip cache. This should hopefully decrease
the execution time of the dependency installation step in each job
because we are no longer going out to pypi on every run and downloading
each dependency individually and can just use the locally cached
version.
* Attempt to increase cache hit rate
* Fix typo
* Fix broken quotes
* Fix another typo
* Add parallel option to docs upload
Right now the compiled docs upload to the artifact store is a bottleneck
for a CI throughput. In an attempt to speed it up this commit sets the
parallel option on the task [1]. This will hopefully reduce the amount
of time spent uploading files to the artifact store.
[1] https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/publish-build-artifacts?view=azure-devops
* Archive docs prior to upload
Parallel upload did not speed up the upload task noticeably. This commit
adds a new step to try compressing the docs prior to uploading. This
should hopefully result in a faster execution time.
* Make z3-solver opt-in for testing
The z3-solver package isn't precompiled on macOS (there is an open issue
about this Z3Prover/z3) so on CI jobs when running on macOS we spend a
great deal of time (typically >10min for each macOS job) building z3 from
source. This is delaying everyone's development and wasting time. This
commit removes z3-solver from the requriements-dev.txt list and manually
installs it on linux and windows CI jobs where precompiled binaries are
available.
* Workaround broken pylint
* Remove unused sequence from unittests
* Fix lint
* Add environment variable to suppress aer warning
* Rename env var and add user config file option
This commit continues where the previous one left off, it makes 2
changes to functionality of the option to suppressing the packaging
warnings. First, it changes the environment variable to be
QISKIT_SUPPRESS_PACKAGING_WARNINGS and applies it to both aer and ibmq.
Then it also adds an option for supressing these warnings to the user
config file. This will enable users to disable the warnings globally
locally without needing to ensure an env var is set. At the same time
the test jobs are updated so that we suppress the warnings there because
we don't install aer in ci (since it's not used).
* Fix lint
* Update qiskit/__init__.py
Co-Authored-By: Kevin Krsulich <kevin@krsulich.net>
* Let env var override user config file
* Adjust logic for printing warnings
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
Co-authored-by: Kevin Krsulich <kevin@krsulich.net>
Co-authored-by: Kevin Krsulich <kevin.krsulich@ibm.com>
* Bump cibuildwheel version and add binary jobs for non-x86
This commit bumps the cibuildwheel version we use for compiling binary
wheels for upload to pypi to the latest release 1.2.0. As part of that
release support for building manylinux 2014 wheels on non-x86
architectures was added (see joerick/cibuildwheel#273). As part of
bumping the version used 3 new jobs are added to the travis config to
trigger building and uploading wheels to pypi for aarch64, ppc64le,
and s390x.
* DNM: Testing non-x86 builds
* Revert "DNM: Testing non-x86 builds"
This reverts commit 690c5e83ad.
* Remove non-x86 jobs from travis config
* Skip pypy wheel builds
* DNM: Test wheel builds
This is a test commit to be reverted prior to merging the branch. It
enables the wheel builds and disables the pypi uploads.
* Revert "DNM: Test wheel builds"
This reverts commit cf689689d4.
* Bump to latest cibuildwheel release
* Revert "Revert "DNM: Test wheel builds""
This reverts commit b8e48671a8.
* Revert "Revert "Revert "DNM: Test wheel builds"""
This reverts commit df39e479ec.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
The new pip depsolver is coming this summer. [1] Until it's ready
this starts running pip check as part of our CI test jobs to make
sure we're not somehow installing conflicting package versions. We
shouldn't be, but if we are better to know and start
preparing/fixing it now.
[1] https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html
* Add tests executing a circuit on each fake backend
Right now there is nothing in validating that we're able to use all the
fake backends defined in qiskit.test.mock.backends. So when we make
changes to the fake backends (or backends in general) we don't know if
they work until someone goes to use them. This was especially needed
for #4016. During the development of that PR all the tests pass but the
docs builds failed because a fake backend is used in a jupyter-execute
section. This commit addresses the situation by adding a simple test
that just runs a simple deterministic circuit that just has some easy
1q gate run optimizations on the fake backend at each optimization level
(to ensure we use all the backend properties needed for execution). This
will provide us coverage over the fake backends to ensure that they at
least are functional as we make changes to terra.
As part of the development of these tests, 2 missing properties from the
new qobj classes were added because the tests caught that they were
missing. Additionally the properties json file for fake yorktown had to
be updated, the properties there were incorrect when run with aer
(and thus a noise model) all the results were invalid.
Fixes#3735
* Add qiskit-aer to ci jobs
To fully test the fake backend properties we need to use aer to build
a noise model and run the simulation with that. This commit adds
qiskit-aer to the ci configuration so that it is a vailable for these
new tests. In general we don't want to rely on aer (because of the
potential circular dependency) but for this case it's seems worth the
tradeoff.
* Reduce error rates on fake_openpulse_2q backend
When running simulations with a noise model the tests were failing on
fake_openpulse_2q because it was set as being too noisy. This commit
drops the error rates set in the properties so we can run a circuit with
a single x gate and get the expected result.
* Handle simulators properly for fake backends
The fake_qasm_simulator is set as a simulator, but the base
fake_provider class was not setup to handle a backend which was a
simulator. This commit handles this by having the fake_qasm_simulator
return no properties, and also updates the run() method to not try and
build a noise model for a simulator.
* Adjust tests to use fake provider
This commit adjusts the tests to use the fake provider to get a list of
backends instead of scanning the backends module for Fake* objects.
* Change fake_qasm_simulator basis to basic aer's
For fake_qasm_simulator's run() method top work when aer is not
installed it needs to be usable with the basic aer simulator. However,
right now the basis set was closer to aer's. This commit adjusts it to
be basic aer's which is LCD for running in a simulator.
* Cleanup arg name for test
* Add explicit PR trigger to azure pipelines
According to the trigger docs for azure [1] not specifying a pr field in
the config should have an implicit pr trigger on all branches.
However, this default PR trigger has stopped working. This commit adds a
manual pr trigger field in an attempt to workaround this issue. It has
been reported that manually adding the PR trigger will resolve the
issue. [2]
[1] https://docs.microsoft.com/en-us/azure/devops/pipelines/build/triggers?view=azure-devops&tabs=yaml#pr-triggers
[2] https://developercommunity.visualstudio.com/comments/949243/view.html
* Revert "Update mergify config to require travis not azure (#3975)"
This reverts commit f534d904b0. This is no
longer needed because azure-pipelines is working again.
* Revert "Add back full travis config during azure pipelines outage (#3973)"
This reverts commit b50f30c03e.
* Start fixing sphinx warnings
Sphinx warnings almost always indicate a docs bug or formatting issue.
Having a docs build without any warnings is important to ensure the
compiled output from sphinx is as expected. This commit starts the
process of fixing the ~430 warnings emitted by the terra docs. Once all
the warnings are fixed this will add '-W' to the sphinx build command so
that any future warnings introduced into the docs will cause a failure.
* More warning fixes
* Fix name 'Command' is not defined warnings
* Fix 'SchedStyle' and 'Pulse' warnings
* More warning fixes
* More warnings fixes
* Fix some warnings
* Fix More pulse module warnings
* Fix Pulse Lint errors
* Last warning fixes outside release notes
This commit fixes the last warnings emitted by the build except for
release notes warnings.
* Attempt to fix release note warning
* Fix lints
* Fix warning in quantum info
* Fix reno warnings
* Fix lint
* Set -W on sphinx build in docs job
* Fix missing indent
* Ignore cyclic import in scheduler
* Revert "Ignore cyclic import in scheduler"
This reverts commit e3cb2f40e0.
* Ignore cyclic-import in scheduler tests
* Trying to fix cyclic errors
* Fix import errors
* Revert "Trying to fix cyclic errors"
This reverts commit 22709cf75c.
* Revert "Revert "Trying to fix cyclic errors""
This reverts commit 3132258e37.
* Trying to fix cyclic errors of qiskit/pulse/commands/instruction.py
* Try to disable cyclic import lint failures
* Fix stray \ket{} usage
* Update qiskit/circuit/quantumcircuit.py
Co-Authored-By: Kevin Krsulich <kevin@krsulich.net>
* Update Statevector.from_label to use list-table
Co-authored-by: SooluThomas <soolu.elto@gmail.com>
Co-authored-by: Kevin Krsulich <kevin@krsulich.net>
This reverts commit fc9c8f7fa5. A recent
github bug was incorrectly resetting the author of commits to the
committer. This revert is just be done to correct this problem, the
commit will be re-added with the correct author set.
* Start fixing sphinx warnings
Sphinx warnings almost always indicate a docs bug or formatting issue.
Having a docs build without any warnings is important to ensure the
compiled output from sphinx is as expected. This commit starts the
process of fixing the ~430 warnings emitted by the terra docs. Once all
the warnings are fixed this will add '-W' to the sphinx build command so
that any future warnings introduced into the docs will cause a failure.
* More warning fixes
* Fix name 'Command' is not defined warnings
* Fix 'SchedStyle' and 'Pulse' warnings
* More warning fixes
* More warnings fixes
* Fix some warnings
* Fix More pulse module warnings
* Fix Pulse Lint errors
* Last warning fixes outside release notes
This commit fixes the last warnings emitted by the build except for
release notes warnings.
* Attempt to fix release note warning
* Fix lints
* Fix warning in quantum info
* Fix reno warnings
* Fix lint
* Set -W on sphinx build in docs job
* Fix missing indent
* Ignore cyclic import in scheduler
* Revert "Ignore cyclic import in scheduler"
This reverts commit e3cb2f40e0.
* Ignore cyclic-import in scheduler tests
* Trying to fix cyclic errors
* Fix import errors
* Revert "Trying to fix cyclic errors"
This reverts commit 22709cf75c.
* Revert "Revert "Trying to fix cyclic errors""
This reverts commit 3132258e37.
* Trying to fix cyclic errors of qiskit/pulse/commands/instruction.py
* Try to disable cyclic import lint failures
* Fix stray \ket{} usage
* Update qiskit/circuit/quantumcircuit.py
Co-Authored-By: Kevin Krsulich <kevin@krsulich.net>
* Update Statevector.from_label to use list-table
Co-authored-by: SooluThomas <soolu.elto@gmail.com>
Co-authored-by: Kevin Krsulich <kevin@krsulich.net>
Right now we're only testing in CI that things work on macOS high sierra
but mojave and catalina have been out for a while and different osx versions
differ enough that it warrants testing the newest version works. This
commit switches the osx jobs in the to use catalina. Additionally, azure pipelines
is removing support for the high sierra environment which means we'll have
to migrate to a newer version anyway.
* Add catalina job
Right now we're only testing in CI that things work on macOS high sierra
but catalina has been out for a while and different osx versions differ
enough that it warrants testing that the new version works too. This
commit adds a new catalina job that runs only python 3.7 to verify that
things work on newer osx.
* Update azure-pipelines.yml
Co-Authored-By: Kevin Krsulich <kevin@krsulich.net>
* Use Mojave for all second stage osx jobs
With the addition of python 3.8 jobs we're running at our job
concurrency limit. To avoid making the run time of ci too long by having
to wait on every commit this commit switches the second stage osx jobs
to use mojave instead of adding duplicated jobs on multiple osx
versions. We'll still use high sierra for the first stage's 3.7 testing
to maintain coverage there.
* Pivot to macos 10.15 catalina
Catalina images have been available on azure pipelines for a few days
now. This commit pivots to using those instead of 10.14 mojave which
have issues with segfaults because of system libraries.
* Use conda for macOS jobs
The pip installable numpy is causing segfaults on catalina and mojave
environments. To workaround this issue this commit switches to using a
conda install for the macos jobs only.
* Install scipy via conda too
* Switch back to 10.14 again
10.15 fails in the same way with or without conda, we've tried 10.14
without conda but not with it. This commit switches back to 10.14 now
that we've switched to using conda to see if the tests pass.
* Switch back to 10.15 without conda
Now that we have a fix for the random segfaulting in the quantum info
tests there shouldn't be a reason we need to keep running in conda. At
the same time this switches back to 10.15 to test the newest version of
macOS. This only reverted back to 10.14 briefly to see if the segfault
was isolated to 10.15, which it was not.
Co-authored-by: Kevin Krsulich <kevin@krsulich.net>
Since we've deprecated python 3.5 having 3.5 be the first round of
testing performed in CI feels out of place. This commit makes the
default jobs run in the first stage of the CI pipeline in azure all use
python 3.7. We still need to verify that 3.5 works and still gate on the
3.5 job passing. But, it doesn't get as much attention now that it's
deprecated. At the same time this commit takes the opportunity to remove
the duplicated python 3.8 job in the travis config. This was added
originally because travis had 3.8 support well before azure pipelines
but was never removed after azure added their 3.8 support.
Co-authored-by: Kevin Krsulich <kevin@krsulich.net>
* Add python 3.8 support and deprecate python 3.5
This commit adds support for running terra under python 3.8. It does
this by adding the trove classifiers to the package metadata and adding
a test job for python 3.8 (only travis for now this can be expanded in
the future when azure pipelines adds it to their image). At the same
time this commit starts the deprecation window for python 3.5. Python
3.5 goes end of life by upstream python in Sept. 2020. We should give
our users running with python 3.5 (which is about 10% of our users based
on pypi data) fair notice that when upstream python stops supporting it
we do as well.
* Fix lint
* Add back empty cache detection
As part of the refactors the empty stestr timing cache removal code was
removed from the travis config, but the cache wasn't removed. Adding the
python 3.8 job causes a failure because travis has no cached timing data
but still creates the empty directory which triggers
mtreinish/stestr#266. This adds back the empty cache removal to
workaround it so we can use the timing data for scheduling in future
runs.
* Add azure-pipelines python 3.8 jobs too
* Fix python 3.8 dictionary keys changed error
Starting in python 3.8 a new RuntimeError is raised, RuntimeError:
dictionary keys changed during iteration. This is caused by modifying an
iterator while looping over it. We were doing that in the
optimize_swap_before_measure pass. This commit fixes this by wrapping
the iterator in list() to make a copy of it for looping. This way we
don't modify the contents of what we're iterating over in the pass.
* Revert "Add azure-pipelines python 3.8 jobs too"
The missing matplotlib wheels are blockers for windows and osx
environment. Since we do not have the necessary dependencies installed
in those ci envs to compile matplotlib from source. We'll rely on just
travis for 3.8 testing until the matplotlib 3.2.0 release is pushed so
we don't have to compile it.
This reverts commit 40157621b1.
* Add skip for failing matplotlib test
The image comparison tests which are quite flaky are failing with
matplotlib compiled on 3.8. Looking at the image output from these
failures are very subtle (looks like resolution differences) again
questioning the value of these tests. This commit just skips the test
that is failing here to unblock the PR.
* Add azure 3.8 test jobs
* Pin to pre-release mpl for python 3.8
* Add spawn guard for python3.8 osx in examples
When running examples with execute, transpile, or any other calls using
parallel_map() the new default for spawn instead of fork requires that
scripts have a __name__ == "__main__" check in them to function
properly. If not calls to parallel_map fail errors around the
bootstrapping phase. To avoid this in the ci jobs this commit adds the
necessary checks to the example scripts we run in ci as part of
test_examples.
* Remove cryptography pin from constraints file
We pinned the cryptography package version back during the cryptography
2.6 release which broke our ci. It was a temporary step to avoid a
packaging issue in CI that was blocking development. However, when the
issue was resolved we never circled back to fix the issue. Now trying to
enable python 3.8 support to terra cryptography >=2.8 is needed for
python 3.8 on windows (to get a precompiled binary). This commit removes
the unecessary pin to unblock windows 3.8 ci.
* Bump cibuildwheel version to build 3.8 wheels
cibuildwheel 1.0.0 was released in early November [1] and added support
for building python 3.8 wheels. This commit bumps the cibuildwheel
version we use in the wheel build jobs at release time to also build 3.8
wheels for upload to pypi.
* Add skip and release note about macos py38 issues
This commit adds a skip for the failing python 3.8 test on osx and
windows so that we don't block everything over a small issue in the
tests. It also add a release note documenting the limitation with python
3.8 on macos with regardess to parallel_map/multiprocessing. Since this
limitation is new for this release (being the first release with python
3.8 support) we should document it as a known issue in the release
notes, especially since it likely won't be resolved until a python 3.8.1
release.
* Revert "Add spawn guard for python3.8 osx in examples"
While we can fix the tests to work on osx python3.8 by adjusting the
example scripts to only call functions using parallel_map from inside a
block run via if __name__ == '__main__': this unecessarily changes the
scripts for the quirks of a single environment. This commit reverts the
example scripts back to their original form and instead just skips the
unittest that executes them on python 3.8 macOS. We already have
documented this limitation in the release notes. When/if we have an
alternative solution for how we launch additional processes in python
3.8 on macOS that does not require this workaround we can look at
removing the skip.
This reverts commit 76ae197631.
* Apply suggestions from code review
Co-Authored-By: Kevin Krsulich <kevin@krsulich.net>
This commit fixes several issues that cropped up during the release jobs
for the 0.10.0 release. All the jobs failed at release time and did not
upload anything to pypi (if they even successfully ran). This commit
fixes all the issues identified so far during that process.
Matthew Treinish