* Reconfigure CI pipelines for PR and merge queue
This commit refactors the pipeline organisation of our CI to reduce the
total number of jobs run on any given PR merge, and to reduce the
critical path length of merging. In particular:
- the lint, docs and QPY tests are combined into one job for a single CI
worker. These three jobs individually add up to less than a test run,
and moving QPY from the first-stage PR test run to this rebalances the
(now) two jobs in the stage to be more equal in runtime.
- a new pipeline is added specifically for the merge queue. Previously
this reused the same two-stage PR pipeline. The two-stage system
unnecessarily lengthened the critical path, as when a PR enters the
merge queue, it as already passed PR CI, and so is highly likely to
pass all jobs. In addition to flattening to a single stage, one each
of the macOS and Windows jobs are removed to lower the amount of VMs
needed and reduce the chances of a timeout (these OSes are more likely
to get a dodgy VM and time out than Linux). The only way a new test
failure should appear (other than a flaky test) is by a logical merge
conflict, which would be quite unlikely to only affect a particular
Python version.
To make it easier to run the lint and docs jobs together, and ensure
that both run even if there's a failure in one, the full lint
configuration is merged back to `tox.ini`, and that's used to do the
linting. This makes it more consistent for developers, as well.
* Allow cargo as an external in 'tox'
* Add display name to lint job
* Use editable install for tox lint job
The lint job will fail unless the Rust components are installed into the
source directory, since that's where `pylint` will look for them to
resolve the imports.
* Fix typo in merge-queue stage condition
* Add support for Python 3.12
Python 3.12.0 was released on 10-02-2023, this commit marks the start of
support for Python 3.12 in Qiskit. It adds the supported Pythonv ersion
in the package metadata and updates the CI configuration to run test
jobs on Python 3.12 and build Python 3.12 wheels on release.
Fixes: #10887
* Add release note
* Avoid deprecated `datetime.datetime.utcnow()` usage
In Python 3.12 `datetime.datetime.utcnow()` has been deprecated, being
replaced by: `datetime.datetime.now(datetime.UTC)`. This commit updates
the usage of `utcnow()` to follow the new convention.
* Adjust UTC usage to support Python 3.8
The recommended alternative for using utcnow() in the deprecation
warnings emitted by Python 3.12 are not compatible with Python 3.8. The
datetime.UTC alias was not added to Python until Python 3.11. To ensure
that the code is compatible with Python < 3.11 this commit updates all
the usage of datetime.UTC to use datetime.timezone.utc instead, which is
what datetime.UTC aliases to in Python >=3.11.
* Pivot to gha for wheel builds and pypi trusted publishers
This commit pivots the wheel publishing jobs to PyPI trusted publishers.
This mechanism authorizes the repository action directly so that user
creds or tokens are not needed to push the wheels anymore. This
mechanism will be more robust as the the github repository is linked
directly to the pypi project and not dependent on a single user account.
The one tradeoff required for this is that we must use github actions to
leverage this PyPI feature. So this commit migrates the wheel publishing
jobs from azure pipelines to github actions (which we were already using
for non-x86 linux wheels).
* Fix macOS image used in macOS jobs
* Fix nits in metapackage job
* Combine tier 1 platform builds into single upload job
* Fix release jobs post-0.25.1 release
This commit fixes 3 issues identified during the 0.25.1 release. The
first is that the name of the qiskit package build/publish job was
invalid in azure pipelines. The `-` character is not valid and throws an
error when the release is triggered. The second issue is that the qiskit
package job is triggered at the same time as all the qiskit-terra
package build/publish jobs. However, the qiskit package is very quick to
build while the qiskit-terra jobs take some time so qiskit will be
published first and qiskit-terra second. This will result in a small
period when `qiskit` is uninstallable because it depends on a yet
unpublished release of qiskit-terra. To address this the build artifact
deployment is split into 2 stages, the first builds the qiskit-terra
precompiled binary wheels and publishes those to pypi first. The second
stage builds the qiskit-terra sdist and the qiskit package. This means
users will always have a working install when they do `pip install
qiskit`. The final fix is the path for the qiskit package's build
artifacts was incorrect. This caused the job to fail because it could
not upload the artifacts for local download or to pypi.
* Fix scoping for dependsOn
* Move metapackage shim for combined releases
Now that Qiskit 0.44.0 has been released, the Qiskit project is now what
was formerly qiskit-terra. However, because Python packaging lacks a
clean mechanism to enable one package superseding another we're not able
to stop shipping a qiskit-terra package that owns the qiskit python
namespace without introducing a lot of potential user friction. So
moving forward the qiskit project will release 2 packages an inner
qiskit-terra which still contains all the library code and public facing
qiskit package which installs that inner package only. To enable this
workflow this commit migrates the metapackage setup.py into the terra
repository and setups build automation to publish a qiskit package in
addition to the inner terra package on each release tag.
* some follow up on https://github.com/Qiskit/qiskit-terra/pull/10530 (#19)
* some follow up on https://github.com/Qiskit/qiskit-terra/pull/10530
* extend some badges
* This Qiskit contains the building blocks for creating and working with quantum circuits, programs, and algorithms. -> This framework allows for building, transforming, and visualizing quantum circuits.
* The explanation of the Bell state is moving to IBM Quantun learning platform
* I think examples/ should eventually be replaced by proper tutorials
* Add StackOverflow as a forum
* Remove link to https://github.com/Qiskit/qiskit-tutorials
* Update README.md
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
* Update README.md
* Update README.md
* broken lines in badges
* doi
---------
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
* Expand lint checks to entire qiskit_pkg dir
* Unify extras in terra setup.py
* Update CI lint job
* Remove unused json imports
* Cleanup manifest file
* Finish comment
---------
Co-authored-by: Luciano Bello <bel@zurich.ibm.com>
* Add GitHub Actions documentation-deployment pipeline
This brings a documentation-deployment pipeline into the Qiskit/Terra
repository, allowing it to fully deploy its documentation to
`qiskit.org`, a task previously only the metapackage could perform.
This does not fully unify the documentation with files from the
metapackage, it just adds a pipeline to do the final deployment.
This includes a revitalised translatable-strings pipeline, which was
previously broken on the metapackage for the last month or two. It also
previously included a fair amount of legacy weight that was no longer
relevant.
* Add missing secret insertions
* Improve logic for deployments
This changes the logic for the deployments so that pushes to 'stable/*'
no longer trigger any deployment to qiskit.org. Instead, tag events
trigger a deployment to the relevant stable branch, and a tag event of
the _latest_ tag triggers a deployment to the documentation root.
The translatables logic is modified to push only the latest full-release
tag.
* Add no-optionals and full-optionals test runs
This restructures the CI slightly to perform a complete "no-optionals"
run, and a complete "all optionals" run (for the optionals that are
readily accessible as Python packages, without complex additional
setup). Previously, we only did a partial test with some of the oldest
optional components, which could have allowed for behaviour to
accidentally require optional components without us noticing.
This splits the `requirements-dev.txt` file into two; lines that remain
in the file are what is actually _required_ to run the test suite,
run the style checks, and do the documentation build. The rest (and
everything that was missing) is added to a new
`requirements-optional.txt` file, which can be used to pull in (almost)
all of the packages that Terra can use to provide additional /
accelerated functionality.
Several tests needed to gain additional skips to account for this
change. There is a good chance that some tests are missing skips for
some libraries that are not the first point of failure, but it's hard to
test explicitly for these in one go.
* Fix typo in coverage workflow
* Try relaxing ipython constraints
* Squash newly exposed lint failures
* Fix typo in tutorials pipeline
* Update the 'also update' comments
* Remove unneeded qiskit-toqm dependency
* Section requirements-optional.txt
* Test all optionals on min not max Python version
Optionals are generally more likely to have been made available on the
older Pythons, and some may take excessively long to provide wheels for
the latest versions of Python.
* Add missing test skip
* Fix optional call
* Use correct boolean syntax
* Fix tests relying on Jupyter
* Install ipykernel in tutorials
* Remove HAS_PDFLATEX skip from quantum_info tests
For simple LaTeX tests, IPython/Jupyter can handle the compilation
internally using MathJax, and doesn't actually need a `pdflatex`
installation. We only need that when we're depending on LaTeX libraries
that are beyond what MathJax can handle natively.
* Include additional tutorials dependencies
* Install all of Terra's optionals in tutorial run
* Do not install all optionals in docs build
* Use class-level skips where appropriate
* Do not install ibmq-provider in tutorials run
* Include matplotlib in docs requirements
* Remove unnecessary whitespace
* Split long pip line
* Only install graphviz when installOptionals
Co-authored-by: Eric Arellano <14852634+Eric-Arellano@users.noreply.github.com>
* Install visualization extras in docs build
* Don't `--upgrade` when installing optionals
This is to prevent any optionals that have a dependency on Terra from
potentially upgrading it to some PyPI version during their installation.
This shouldn't happen with the current development model of having only
one supported stable branch and the main branch, but the `-U` is
unnecessary, and not having it is safer for the future.
* Update secondary installation of Terra in docs build
* Install all optionals during the docs build
I yoyo-ed on this, not wanting it to be too onerous to build the
documentation, but in the end, we need to have the optional features
installed in order to build most of the documentation of those features,
and some unrelated parts of the documentation may use these features to
enhance their own output.
* Fix test setup job
* Remove duplication of matplotlib in requirements files
* Update image test installation command
* Restore editable build
* Move `pip check` to `pip` section
* Remove redundant "post-install" description
* Expand comment on first-stage choices
* pytohn lol
---------
Co-authored-by: Eric Arellano <14852634+Eric-Arellano@users.noreply.github.com>
* Drop support for Python 3.7
This updates our documentation and build processes to set Python 3.8 as
the oldest support version of Python, consistent with our policy on
Python support.
This commit also removes remaining Python-version gating, since we were
principally using this to account for differences between 3.7 and 3.8.
Several backport packages are no longer required for any suppported
Python version, because of this change.
* Remove test for multiprocessing on Mac
With Python 3.7 support now dropped, there are no versions of Python on
macOS that we expect to support with multiprocessing again. There's no
guarantee that the hypothetical Python 10.11 (???) that this test was
previously checking would be any more valid than existing versions.
---------
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
* Attempt to trigger Azure pipelines on merge queue events
This commit attempts to add a trigger condition to our azure pipelines
job to trigger on commits to the gh merge queue branch.
* Only trigger PR CI in queue, not push CI
---------
Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
* Add `rust-toolchain.toml` for a consistent Rust development version
* Add components
* Simplify CI to not set Rust version
It will now use the rust-toolchain.toml file. This is possible because Rustup is already on the PATH, evidenced by us previously running `rustup default`
* Build Azure wheels with stable toolchain
* Use Stable for the sdist test build
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit bumps the minimum supported rust version from 1.56.1 to
1.61.0 (which is the LCD for all our dependencies) which was released
on 2022-05-19. By doing this we're now able to bump all of our
upstream dependencies to use the latest releases.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Revert "[Test] Pin maximum python version in CI to <3.11.1 (#9296)"
This reverts commit 07e0a2fc79.
* Do not treat __init_subclass__ as a special type method
* Release note
* Apply suggestions from code review
Co-authored-by: Julien Gacon <gaconju@gmail.com>
* Use inspect.getattr_static to bypass descriptor call
* Update release note
* Update wrap_method test
* Adjust wording on release note
Co-authored-by: Julien Gacon <gaconju@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add support for Python 3.11
Python 3.11.0 was released on 10-24-2022, this commit marks the start of
support for Python 3.11 in qiskit. It adds the supported Python version in
the package metadata and updates the CI configuration to run test jobs
on Python 3.11 and build Python 3.11 wheels on release.
* Fix inspect.Parameter usage for API change in 3.11
Per the Python 3.11.0 release notes inspect.Parameter now raises a
ValueError if the name argument is a Python identifier. This was causing a
test failure in one case where a parameter named `lambda` was used.
This commit adjusts the parameter name in the tests to be lam to avoid
this issue.
* Set a version cap on the jax dev requirement
Currently jax doesn't publish Python 3.11 wheels which is blocking test
runs with python 3.11. Since jax is an optional package only used for
the gradient package we can just skip it as isn't a full blocker for
using python 3.11. This commit sets an environment marker on the jax
dev requirements to only try to install it on Python < 3.11.
* Set python version cap on cplex in CI
* DNM: Test wheel builds work
* Skip tests on i686/win32 wheel buids with python 3.11
* Revert "DNM: Test wheel builds work"
This reverts commit 725c21b465.
* Run QPY backwards compat tests on trailing edge Python version
This commit moves the qpy backwards compatibility testing from the
leading edge python version, which in this PR branch is Python 3.11, to
the trailing edge Python version which is currently 3.7. Trying to add
support for a new Python version has demonstrated that we can't use the
leading edge version as historical versions of Qiskit used to generate
old QPY payloads are not going to be generally installable with newer
Python versions. So by using the trailing edge version instead we can
install all the older versions of Qiskit as there is Python
compatibility for those Qiskit versions. Eventually we will need to
raise the minimum Qiskit version we use in the QPY tests, when Python
3.9 goes EoL in October 2025 and Qiskit Terra 0.18.0 no longer has any
supported versions of Python it was released for. We probably could
get by another year until Python 3.10 goes EoL in 2026 it just means
we're building 0.18.x and 0.19.x from source for the testing, but when
Python 3.11 becomes our oldest supported version we'll likely have to
bump the minimum version.
This does go a bit counter to the intent of the test matrix to make the
first stage return fast and do a more through check in the second stage.
But, in this case the extra runtime is worth the longer term stability
in the tests.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
The macOS 10.15 image we're using in azure pipelines has been deprecated
for some time (as Apple has dropped support for the version) and they've
started periodic brownouts on the version to accelerate the transition
off of the image. We have stayed pinned at the older version because we
had compatibility issues with the newer releases in the past. But, since
this is no longer an option this commit bumps us one version from 10.15
to 11. This doesn't go straight to 12 as Apple proactively disables
support for older platforms in newer OS releases and in Qiskit we try to
maximize platform support, even those using older Apple hardware, so the
minimal version update is made.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Explicitly set MSRV for rust extension
This commit explicitly sets the MSRV (minimum supported rust version)
for Qiskit Terra to 1.56 which was released in October 2021 (with 1.56.1
which fixed two CVEs being released Nov. 1st 2021). Previously we had
avoided setting a hard MSRV and opted to try and just loosely support the
past 6 months of rust releases. However, managing it loosely has proven
tricky to manage in practice. This commit sets a hard version for MSRV
and modifies a test job to validate we're able to compile with the MSRV.
This should ensure we're able to avoid breaking compatibility for that
version.
* Explicitly set MSRV to 1.56.1 instead of loosely 1.56
* Explicitly set version as variable in CI config
* Assign MSRV to azure variable
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Overhaul Azure Pipelines configuration
This restructures our monolithic `azure-pipelines.yml` file into a
modular setup that separates the logic of specifying "what to run" from
"how to run". For example, the different processes for running the
Python tests on Linux, Mac and Windows are separated out into three
template files, which are then called by the "branch push" and "PR sync"
pipeline files in a few different stages. This is also to centralise
the logic for "how to run"; previously, it was duplicated in a few
different stages.
There are also major changes to content of "what to run".
Branch push:
This now only runs a single Linux test, just as a sanity check.
Previously it ran the full PR CI suite, including lint, docs,
tutorials, and the entire Python testing matrix. This was
unnecessary, as the branch protection rules and PR merge
strategy ensured that the code must already have been up-to-date
and passed the PR CI before it could be pushed to the branch, so
was simply burning CI resources for no benefit. To the best of
my knowledge, the branch-push CI had never caught a bug.
PR sync:
This remains split into two stages, but now only the oldest and
newest Python versions are tested, rather than the entire
matrix. It is exceedingly rare (I'm not sure it's ever
happened) that a commit breaks an intermediate Python version
_only_, so instead we add a new "nightly" CI run to test the
full matrix, which should be sufficient to catch these. The
"preliminary" stage is reduced to lint, docs and a single test
run; about half of PR commits cause a CI failure, and over 90%
of these would be caught by the new structure. The second stage
contains the tutorials and the rest of the
`{oldest,newest},{windows,mac,linux}` matrix. This is to reduce
total CI load, with the intent that failing runs should almost
universally fail just as quickly as they did before.
Nightly:
Entirely new. This is tests of the full matrix of Python
versions and OSes for the `main` and `stable` branches, if a
given branch was updated since the last run. It should be very
unlikely that this run catches errors.
Tag push:
The deployment process is unchanged, just refactored into a
format for easier management.
* Refactor split pipelines into single entry point
This maintains the same logical splits as the previous commit, but
rather than using four separate pipeline files, it uses template
conditional compilation to put them in a single large file. This lets
us keep continuity with our current Azure setup, and lets us easily
share a series of configuration variables (including a general YAML
object) between the different stages.
This commit swaps from using a `strategy: matrix` in the test jobs, in
favour of using templated loops over the variables from the main
`azure-pipelines.yml` file. This is because passing strings like
`"3.10"` contained within YAML arrays from one template file to another
appears to trigger some sort of implicit conversion somewhere, and
`"3.10"` can be interpreted as `3.1`.
* Leave comment on nightly testing failure
* Reinstate autocancel
* Quote string parameters
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Split long environment variable
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Add more comments on YAML syntax
* Add option to install from sdist in Linux tests
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Prepare 0.20.0 release
To prepare for the upcoming 0.20.0 release this commit moves all the
release notes into a 0.20 subdirectory to keep them separate from future
development notes post-release. It also adds a release note with the
prelude section providing the high level overview of the release. When
this merges it should be the commit used to tag the 0.20.0 release.
* Move and update new release notes
* Fix typo
* Update more release notes
* Move and update new release note
* More release note updates
* Drop operation release note
* Add missing release notes
* Fix one more release note
* Fix upgrade and deprecation notes
* Remove duplicate release notes from 0.19.x
* Remove Operation from doc toctree
* Fix more bug fix relaese notes
* Finish first pass over release notes
* Add link to rust in prelude
* Move and update new release notes
* Apply suggestions from code review
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Release note fixes
* Fix macOS arm64 wheel config
* Move and update new release notes
* Revert tox.ini change
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
* Implement multithreaded stochastic swap in rust
This commit is a rewrite of the core swap trials functionality in the
StochasticSwap transpiler pass. Previously this core routine was written
using Cython (see #1789) which had great performance, but that
implementation was single threaded. The core of the stochastic swap
algorithm by it's nature is well suited to be executed in parallel, it
attempts a number of random trials and then picks the best result
from all the trials and uses that for that layer. These trials can
easily be run in parallel as there is no data dependency between the
trials (there are shared inputs but read-only). As the algorithm
generally scales exponentially the speed up from running the trials in
parallel can offset this and improve the scaling of the pass. Running
the pass in parallel was previously tried in #4781 using Python
multiprocessing but the overhead of launching an additional process and
serializing the input arrays for each trial was significantly larger
than the speed gains. To run the algorithm efficiently in parallel
multithreading is needed to leverage shared memory on shared inputs.
This commit rewrites the cython routine using rust. This was done for
two reasons. The first is that rust's safety guarantees make dealing
with and writing parallel code much easier and safer. It's also
multiplatform because the rust language supports native threading
primatives in language. The second is while writing parallel cython
code using open-mp there are limitations with it, mainly on windows. In
practice it was also difficult to write and maintain parallel cython
code as it has very strict requirements on python and c code
interactions. It was much faster and easier to port it to rust and the
performance for each iteration (outside of parallelism) is the same (in
some cases marginally faster) in rust. The implementation here reuses
the data structures that the previous cython implementation introduced
(mainly flattening all the terra objects into 1d or 2d numpy arrays for
efficient access from C).
The speedups from this PR can be significant, calling transpile() on a
400 qubit (with a depth of 10) QV model circuit targetting a 409 heavy
hex coupling map goes from ~200 seconds with the single threaded cython
to ~60 seconds with this PR locally on a 32 core system, When transpiling
a 1000 qubit (also with a depth of 10) QV model circuit targetting a 1081
qubit heavy hex coupling map goes from taking ~6500 seconds to ~720
seconds.
The tradeoff with this PR is for local qiskit-terra development a rust
compiler needs to be installed. This is made trivial using rustup
(https://rustup.rs/), but it is an additional burden and one that we
might not want to make. If so we can look at turning this PR into a
separate repository/package that qiskit-terra can depend on. The
tradeoff here is that we'll be adding friction to the api boundary
between the pass and the core swap trials interface. But, it does ease
the dependency on development for qiskit-terra.
* Sanitize packaging to support future modules
This commit fixes how we package the compiled rust module in
qiskit-terra. As a single rust project only gives us a single compiled
binary output we can't use the same scheme we did previously with cython
with a separate dynamic lib file for each module. This shifts us to
making the rust code build a `qiskit._accelerate` module and in that we
have submodules for everything we need from compiled code. For this PR
there is only one submodule, `stochastic_swap`, so for example the
parallel swap_trials routine can be imported from
`qiskit._accelerate.stochastic_swap.swap_trials`. In the future we can
have additional submodules for other pieces of compiled code in qiskit.
For example, the likely next candidate is the pauli expectation value
cython module, which we'll likely port to rust and also make parallel
(for sufficiently large number of qubits). In that case we'd add a new
submodule for that functionality.
* Adjust random normal distribution to use correct mean
This commit corrects the use of the normal distribution to have the mean
set to 1.0. Previously we were doing this out of band for each value by
adding 1 to the random value which wasn't necessary because we could
just generate it with a mean of 1.0.
* Remove unecessary extra scope from locked read
This commit removes an unecessary extra scope around the locked read for
where we store the best solution. The scope was previously there to
release the lock after we check if there is a solution or not. However
this wasn't actually needed as we can just do the check inline and the
lock will release after the condition block.
* Remove unecessary explicit type from opt_edges variable
* Fix indices typo in NLayout constructor
Co-authored-by: Jake Lishman <jake@binhbar.com>
* Remove explicit lifetime annotation from swap_trials
Previously the swap_trials() function had an explicit lifetime
annotation `'p` which wasn't necessary because the compiler can
determine this on it's own. Normally when dealing with numpy views and a
Python object (i.e. a GIL handle) we need a lifetime annotation to tell
the rust compiler the numpy view and the python gil handle will have the
same lifetime. But since swap_trials doesn't take a gil handle and
operates purely in rust we don't need this lifetime and the rust
compiler can deal with the lifetime of the numpy views on their own.
* Use sum() instead of fold()
* Fix lint and add rust style and lint checks to CI
This commit fixes the python lint failures and also updates the ci
configuration for the lint job to also run rust's style and lint
enforcement.
* Fix returned layout mapping from NLayout
This commit fixes the output list from the `layout_mapping()`
method of `NLayout`. Previously, it incorrectly would return the
wrong indices it should be a list of virtual -> physical to
qubit pairs. This commit corrects this error
Co-authored-by: georgios-ts <45130028+georgios-ts@users.noreply.github.com>
* Tweak tox configuration to try and reliably build rust extension
* Make swap_trials parallelization configurable
This commit makes the parallelization of the swap_trials() configurable.
This is dones in two ways, first a new argument parallel_threshold is
added which takes an optional int which is the number of qubits to
switch between a parallel and serial version. The second is that it
takes into account the the state of the QISKIT_IN_PARALLEL environment
variable. This variable is set to TRUE by parallel_map() when we're
running in a multiprocessing context. In those cases also running
stochastic swap in parallel will likely just cause too much load as
we're potentially oversubscribing work to the number of available CPUs.
So, if QISKIT_IN_PARALLEL is set to True we run swap_trials serially.
* Revert "Make swap_trials parallelization configurable"
This reverts commit 57790c84b0. That
commit attempted to sovle some issues in test running, mainly around
multiple parallel dispatch causing exceess load. But in practice it was
broken and caused more issues than it fixed. We'll investigate and add
control for the parallelization in a future commit separately after all
the tests are passing so we have a good baseline.
* Add docs to swap_trials() and remove unecessary num_gates arg
* Fix race condition leading to non-deterministic behavior
Previously, in the case of circuits that had multiple best possible
depth == 1 solutions for a layer, there was a race condition in the fast
exit path between the threads which could lead to a non-deterministic
result even with a fixed seed. The output was always valid, but which
result was dependent on which parallel thread with an ideal solution
finished last and wrote to the locked best result last. This was causing
weird non-deterministic test failures for some tests because of #1794 as
the exact match result would change between runs. This could be a bigger
issue because user expectations are that with a fixed seed set on the
transpiler that the output circuit will be deterministically
reproducible.
To address this is issue this commit trades off some performance to
ensure we're always returning a deterministic result in this case. This
is accomplished by updating/checking if a depth==1 solution has been
found in another trial thread we only act (so either exit early or
update the already found depth == 1 solution) if that solution already
found has a trial number that is less than this thread's trial number.
This does limit the effectiveness of the fast exit, but in practice it
should hopefully not effect the speed too much.
As part of this commit some tests are updated because the new
deterministic behavior is slightly different from the previous results
from the cython serial implementation. I manually verified that the
new output circuits are still valid (it also looks like the quality
of the results in some of those cases improved, but this is strictly
anecdotal and shouldn't be taken as a general trend with this PR).
* Apply suggestions from code review
Co-authored-by: georgios-ts <45130028+georgios-ts@users.noreply.github.com>
* Fix compiler errors in previous commit
* Revert accidental commit of parallel reduction in compute_cost
This was only a for local testing to prove it was a bad idea and was
accidently included in the branch. We should not nest the parallel
execution like this.
* Eliminate short circuit for depth == 1 swap_trial() result
This commit eliminates the short circuit fast return in swap_trial()
when another trial thread has found an ideal solution. Trying to do this
in a parallel context is tricky to make deterministic because in cases
of >1 depth == 1 solutions there is an inherent race condition between
the threads for writing out their depth == 1 result to the shared
location. Different strategies were tried to make this reliably
deterministic but there wa still a race condition. Since this was just a
performance optimization to avoid doing unnecessary work this commit
removes this step. Weighing improved performance against repeatability
in the output of the compiler, the reproducible results are more
important. After we've adopted a multithreaded stochastic swap we can
investigate adding this back as a potential future optimization.
* Add missing docstrings
* Add section to contributing on installing form source
* Make rust python classes pickleable
* Add rust compiler install to linux wheel jobs
* Try more tox changes to fix docs builds
* Revert "Eliminate short circuit for depth == 1 swap_trial() result"
This reverts commit c510764a77. The
removal there was premature and we had a fix for the non-determinism in
place, ignoring a typo which was preventing it from working.
Co-Authored-By: Georgios Tsilimigkounakis <45130028+georgios-ts@users.noreply.github.com>
* Fix submodule declaration and module attribute on rust classes
* Fix rust lint
* Fix docs job definition
* Disable multiprocessing parallelism in unit tests
This commit disables the multiprocessing based parallelism when running
unittest jobs in CI. We historically have defaulted the use of
multiprocessing in environments only where the "fork" start method is
available because this has the best performance and has no caveats
around how it is used by users (you don't need an
`if __name__ == "__main__"` guard). However, the use of the "fork"
method isn't always 100% reliable (see
https://bugs.python.org/issue40379), which we saw on Python 3.9 #6188.
In unittest CI (and tox) by default we use stestr which spawns (not using
fork) parallel workers to run tests in parallel. With this PR this means
in unittest we're now running multiple test runner subprocesses, which
are executing parallel dispatched code using multiprocessing's fork
start method, which is executing multithreaded rust code. This three layers
of nesting is fairly reliably hanging as Python's fork doesn't seem to
be able to handle this many layers of nested parallelism. There are 2
ways I've been able to fix this, the first is to change the start method
used by `parallel_map()` to either "spawn" or "forkserver" either of
these does not suffer from random hanging. However, doing this in the
unittest context causes significant overhead and slows down test
executing significantly. The other is to just disable the
multiprocessing which fixes the hanging and doesn't impact runtime
performance signifcantly (and might actually help in CI so we're not
oversubscribing the limited resources.
As I have not been able to reproduce `parallel_map()` hanging in
a standalone context with multithreaded stochastic swap this commit opts
for just disabling multiprocessing in CI and documenting the known issue
in the release notes as this is the simpler solution. It's unlikely that
users will nest parallel processes as it typically hurts performance
(and parallel_map() actively guards against it), we only did it in
testing previously because the tests which relied on it were a small
portion of the test suite (roughly 65 tests) and typically did not have
a significant impact on the total throughput of the test suite.
* Fix typo in azure pipelines config
* Remove unecessary extension compilation for image tests
* Add test script to explicitly verify parallel dispatch
In an earlier commit we disabled the use of parallel dispatch in
parallel_map() to avoid a bug in cpython associated with their fork()
based subprocess launch. Doing this works around the bug which was
reliably triggered by running multiprocessing in parallel subprocesses.
It also has the side benefit of providing a ~2x speed up for test suite
execution in CI. However, this meant we lost our test coverage in CI for
running parallel_map() with actual multiprocessing based parallel
dispatch. To ensure we don't inadvertandtly regress this code path
moving forward this commit adds a dedicated test script which runs a
simple transpilation in parallel and verifies that everything works as
expected with the default parallelism settings.
* Avoid multi-threading when run in a multiprocessing context
This commit adds a switch on running between a single threaded and a
multithreaded variant of the swap_trials loop based on whether the
QISKIT_IN_PARALLEL flag is set. If QISKIT_IN_PARALLEL is set to TRUE
this means the `parallel_map()` function is running in the outer python
context and we're running in multiprocessing already. This means we do
not want to be running in multiple threads generally as that will lead
to potential resource exhaustion by spawn n processes each potentially
running with m threads where `n` is `min(num_phys_cpus, num_tasks)` and
`m` is num_logical_cpus (although only
`min(num_logical_cpus, num_trials)` will be active) which on the typical
system there aren't enough cores to leverage both multiprocessing and
multithreading. However, in case a user does have such an environment
they can set the `QISKIT_FORCE_THREADS` env variable to `TRUE` which
will use threading regardless of the status of `QISKIT_IN_PARALLEL`.
* Apply suggestions from code review
Co-authored-by: Jake Lishman <jake@binhbar.com>
* Minor fixes from review comments
This commits fixes some minor details found during code review. It
expands the section on building from source to explain how to build a
release optimized binary with editable mode, makes the QISKIT_PARALLEL
env variable usage consistent across all jobs, and adds a missing
shebang to the `install_rush.sh` script which is used to install rust in
the manylinux container environment.
* Simplify tox configuration
In earlier commits the tox configuration was changed to try and fix the
docs CI job by going to great effort to try and enforce that
setuptools-rust was installed in all situations, even before it was
actually needed. However, the problem with the docs ci job was unrelated
to the tox configuration and this reverts the configuration to something
that works with more versions of tox and setuptools-rust.
* Add missing pieces of cargo configuration
Co-authored-by: Jake Lishman <jake@binhbar.com>
Co-authored-by: georgios-ts <45130028+georgios-ts@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Use manylinux2014 on Linux
Numpy and Scipy are moving to drop manylinux2010 wheels on newer
platforms, which gives us some cover to do the same. To test, we need
to ensure that we pull Numpy and Scipy in binary form only (forcing pip
to install slightly older versions like 1.21 instead of 1.22 on
unsupported Python versions), rather than attempting to build Numpy from
source as part of our testing process.
* Bump cibuildwheel version
This commit tweaks the apt usage in the Ubuntu CI job configuration.
Previously the Ubuntu CI jobs were using 'apt' which emits a warning
about having an unstable CLI. This commit switches to use apt-get
to avoid the warning and calls apt-get update prior to the installation
step to ensure we've synced with the mirror prior to installing.
* Add support for Python 3.10
Python 3.10.0 was released on 10-04-2021, this commit marks the support
of Python 3.10 in qiskit-terra. It adds the supported python version in
the package metadata and updates the CI configuration to run test jobs
on Python 3.10 and build Python 3.10 wheels.
* Fix typo
* Update default envlist in tox.ini to include 3.10
* Bump cibuildwheel to the latest version
This also takes the opportunity to deduplicate the cibuildwheel
configuration using the pyproject.toml support in newer versions of
cibuildwheel. The common options for all builds are put there and per
build overrides (which are only for cross compiling arm wheels) are left
as environment variables in the CI configuration.
* Add missing cibuildwheel config to pyproject.toml
* Ignore internal deprecation warning emitted by jupyter in ci
* Fix black
Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit fixes the VM image we use in azure pipelines for the release
CI jobs that run on linux. For some reason (likely just a copy and
paste error that never was corrected) previously these jobs were set to
run on Ubuntu 16.04 which is end of life now and no longer available on
azure pipelines. This was the source of an error during the 0.19.0
release as when CI went to run the job it failed because it couldn't
find the requested image. This commit fixes this issue by switching the
jobs to use 'ubuntu-latest' which will also be the latest available
ubuntu lts release on azure pipelines. Which version we run doesn't
really matter because the wheel jobs run in docker and the sdist jobs
just run python to create an sdist (which just creates a tarball of the
project's source code).
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Drop support for Python 3.6
As has been advertised for some time the Qiskit 0.19.0 release is the
last release which supports Python 3.6 (which goes End of Life in
Decemeber). This commit drops support accordingly so that the minimum
supported Python version is 3.7.
* Add release note
* Fix compatibility with Matplotlib 3.5
The problem was `qiskit.visualization.bloch.Arrow3D`, which subclassed
`matplotlib.patches.FancyArrowPatch` (a 2D patch), but not the 3D
machinery. It is now made an unholy multiple-inherited abomination of
both the 2D patch and `mpl_toolkits.mplot3d.art3d.Patch3D`; the latter
is a relatively thin wrapper (in terms of attributes) around the 2D
patch, so this is not too terrible.
Matplotlib 3.5 calls the `Patch3D.do_3d_projection` method using a
deprecated parameter, triggering two warnings, unless the artist's
module appears to have come from `mpl_toolkits.mplot3d.art3d`, even if
the new calling convention is respected. The warning is only triggered
when the figure is drawn, which may well be outside of our control, so
we cannot suppress the warnings. Instead, we just lie about the module
the arrow patch was defined in, to trick it into not warning, because we
use the new calling convention. This is supported at least as far back
as Matplotlib 3.3, which is the current minimum supported version. The
nasty hack should be removable once Matplotlib 3.6 is the minimum
version because the deprecation period will expire.
* Remove useless word
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Pin matplotlib <3.5
This is temporary to get CI running again, while we prepare a proper
fix for the latest version.
* Remove unused imports
A bug in pylint can sometimes cause unused imports in certain files to
be ignored if they're used in a module-level comment type hint somewhere
that's imported first. For more, see:
https://github.com/Qiskit/qiskit-terra/pull/7270/#discussion_r749854451
* Pin matplotlib in setup.py too
* Pin matplotlib in azure-pipelines.yml
* Bump minimum matplotlib version to 3.3.0
In matplotlib 3.4.0 released on 03/26/2021 there were some deprecations
around our use of Axes3d in the state visualization and bloch sphere
modules. Fixing our usage to avoid these deprecations was previously
attempted in #6136 and #6087 but we did not move forward there because
the supported API we need to use matplotlib 3.3.0 when it was
introduced. We previously did not want to raise our minimum supported
matplotlib version to 3.3.0 to accomodate this. However, recent changes
in transitive dependency of mpl, pyparsing, caused an incompatibility
when running with older mpl versions around mathtex. To fix that we
had to pin pyparsing in the constraints file (see #7174). It appears
that now is a better time to raise our minimum version because the
burden of trying to keep support for older matplotlib versions is higher
than it once was.
This commit bumps the minimum matplotlib version to 3.3.0, updates the
usage of Axes3d to avoid the deprecated usage in matplotlib 3.4.0, and
removes the contraints pinning introduced in #7174. This should get us
on supported releases for matplotlib moving forward and it'll hopefully
be a while before we encounter this kind of version issue in the future.
Fixes#6136
* Run black
* Restore constraints pinning
I was under the mistaken impression that the current mpl releases were
compatible with pyparsing3, but looking at the linked mpl issue it's not
fixed yet only for their development so far. So to unblock CI this
restores the pinning as it's still needed until mpl releases a fix.
* Fix matplotlib 3.3.x compat
* Fix version string comparison typing
* Fix typo
* Fix second `ax` typo
* Fix gatemap tests with newer matplotlib
* Fix lint
Co-authored-by: Jake Lishman <jake@binhbar.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Currently, the images are just set to `macOS-latest`, which resolves to
10.15, so this has no immediate change in behaviour. However, the
GitHub/Azure images will soon be moving `macOS-latest` to 11.x, and
those images do not have built-in Python 3.6 support. Terra 0.19 will
still support Python 3.6, so we need to maintain a suitable testing
environment for it, and with Terra being (mostly) a Python-only package,
the 11.x/10.15 difference should not be as important as the different
Python versions.
See actions/virtual-environments#4060.
In the linux py37 CI job ever since #6175 we build an sdist and then
install that in a venv manually in a test /tmp directory we create for
running the tests. However, as part of that we were handling the .stestr
history caching incorrectly. We cache the .stestr directory's times.dbm
files (which contain a record of the most recent run time for each test)
to provide scheduler hints to stest in future runs. But, instead of
updating the .stestr cache with the data from the run we were just
re-populating the data we pulled from the cache. This has also been
somewhat error prone because in some cases the cache isn't downloaded
correctly and will cause an error when we try to remove unecessary files
from the cache and the .stestr directory doesn't exist, presumably
because azure didn't pull it from the cache correctly. This commit fixes
the issue by updating the generate results post processing step to
correctly copy the most recent data to the cached path after the run.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
After 0cdacec (#6865) sped up the docs build to a reasonable time again,
they are no longer a CI bottleneck. This splits the last CI stage
between the other two (docs in the preliminary stage, tutorials in the
main tests), increasing the parallelisation of the stages, and hopefully
getting total CI times down quite a lot.
* Consolidate quick CI jobs into 3.7 linux unittest job
This commit consolidates the 2 fast executing CI jobs for the QPY
compatibility testing and image comparison test validation to be stages
at the end of the linux 3.7 CI job. Our primary resource constraint in
CI is the number of concurrent jobs we can run (which is currently 10
jobs). These two jobs typically take < 3minutes total which includes
provision time, git checkout, and pip dependency installation time.
Since these jobs execute so quickly there is no reason to waste a CI
node to run them when we can just append these to the end of an existing
job. This commit makes this change and adds runs these test job as
separate stages at the end of the linux 3.7 job. This should hopefully
reduce the resource contention in CI slightly and improve CI throughput.
* Fix typo
* Add QPY backwards compatibility testing
This commit adds a new test job to test backwards compatibility in QPY.
One of the guarantees we have for qpy as a format is that current qiskit
can load a file generated by an older version of qiskit. However it's
not much of a guarantee without testing it holds true. This commit adds
a quick CI job that will install every old release of qiskit since qpy
was introduced of qiskit generate some qpy files and then load them with
the current version of qiskit's main branch with any proposed PRs to verify
that we don't accidently merge something that breaks the backwards
compatibility of the format. Overall these tests are extremely quick (we
could even add this onto an existing job if people prefer) where it takes
~1 sec to generate the qpy files and ~2 seconds to load the files. So even
as the number of releases grow we don't have too much about job runtime.
* Fix whitespace error
* Fix bash syntax
* Add --version flag to test_qpy.py script
This commit adds a --version flag to the test_qpy.py script which is
used to specify the version of qpy being tested. It will trigger the
generation of additional test circuits to verify that new features that
are supported in qpy are tested and we don't only validate the 0.18.0
baseline capabilities.
* Update test/qpy_compat/run_tests.sh
Co-authored-by: Jake Lishman <jake@binhbar.com>
* Verify the generation script works with current HEAD
This commit adds a last step to the qpy compat test bash script to run
the generation and test the loading with the current dev branch. While
we have validation of this qpy path via the unittests we don't exercise
there isn't a regression in the compat tests script run here. By doing
this last step we can catch a potential regression in the test job.
Co-authored-by: Jake Lishman <jake@binhbar.com>
* wip: initial move
* fix a bug when no coupling_map is supplied
* Improved documentation and parameter settings
* Improved logical/physical qubit mapping
* Better handling of measurements and final layout
* minor fixes
* add unit tests
* wip: start refactoring
* improve layout handling
* simplify dag construction and fix to output all ops
* fix to work with ApplyLayout
* focus on layout+routing
* improve how to call MIPMapping
* simplify the spec
* add an error handling
* fix how to call MIPMapping pass
* update user warning for change of given layout
* wip: refactor mip model
* improve top-level interface
* return original dag if fails to solve MIP
* minor fixes
* care the case no cplex is installed
* fix not to ignore swaps in original layers
* fix mismatch in layer and su4layer indices
* simplify by focusing only on depth objective
* rename MIP to BIP
* lint
lint
lint
lint
* improve interface around dummy steps
* improve error message when cplex fails
* restore level3 preset passmanager
* improve depth objective
* Rework optional dependency and add test configuration
This commit reworks the logic around the optional dependency on cplex.
The pass despite raising an exception if cplex wasn't installed was
unconditionally trying to import cplex from the model module at import
time. To avoid failures from that this adds a try block around the
actual cplex usage in the model module and exports whether it's
available or not. At the same time this is leveraged to add a skip
condition on the tests so we only run them if cplex is installed. Then
to run tests in CI we install cplex in the linux 3.7 job so we at least
run it in one environment. It's not available in all our supported
environments so getting full coverage in every env is not worth trying
for, so just testing the qiskit portion works is enough.
* Fix lint
* Fix test, lint and logging
* Fix lint and test, update docstring
* Fix docstring and improve performance taking the risk of failing to map
* Update docstring
* Replace cplex.Cplex with docplex.mp.model.Model to improve readability
Replace cplex.Cplex with docplex.mp.model.Model to improve readability
fix requiring docplex version
move docplex to requirements-dev
* Improve BIPMappingModel interface
simplify and improve docstring
Improve BIPMappingModel interface
* Update qiskit/transpiler/passes/routing/algorithms/bip_model.py
Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>
* Fix docplex dependency
* fix unnecessary-comprehension
* lint
* lint
* update how to set cplex params and remove redundant constraints
* skip unless docplex
* fix a bug in arg threads
* remove python version restriction
* Change spec to require the number of virtual and physical qubits are the same
* Update to use 'requires' to require precondition passes
* Update qiskit/transpiler/passes/routing/algorithms/bip_model.py
Co-authored-by: Luciano Bello <bel@zurich.ibm.com>
* lint
* Add release note
Add release note
fix typos
* Update docstring
Update docstring
fix typo
* Improve error handling
* Change to return mapped dag for circuits with only 1q-gates
* Revert the spec change in the case of only 1q-gates
* Update releasenotes/notes/add-bip-mapper-f729f2c5672d7f3e.yaml
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Update releasenotes/notes/add-bip-mapper-f729f2c5672d7f3e.yaml
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Update qiskit/transpiler/passes/routing/bip_mapping.py
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Update qiskit/transpiler/passes/routing/bip_mapping.py
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Update qiskit/transpiler/passes/routing/bip_mapping.py
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Update qiskit/transpiler/passes/routing/bip_mapping.py
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
Co-authored-by: Giacomo Nannicini <gnannicini@users.noreply.github.com>
Co-authored-by: Giacomo Nannicini <giacomo.n@gmail.com>
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>
Co-authored-by: Luciano Bello <bel@zurich.ibm.com>
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit updates the azure pipelines configuration for the tutorials
job. There was a todo in there from right before the 0.17.0 release
about using git versions of ignis and aqua which we were supposed to
have fixed after the release so we don't lose the backwards compat
testing the tutorials job provides. Since then we've also removed the
aqua usage from qiskit-tutorials as the general algorithms and opflow
tutorials have been updated to use terra and the applications tutorials
have been migrated to the application repositories. So this commit
removes aqua from the installed packages list.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
During the 0.17.2 release the windows wheel builds failed during the
setup phase of the job. This was because we were needlessly installing
the vcpython27 package as part of the job setup. This was only needed if
we were building python 2.7 wheels but was blindly copied from the
examples in the cibuildwheel documentation. However, since the job was
originally setup that package no longer exists and was causing a job
failure. This commit fixes the ci job by just removing this unecessary
step. It also takes the opportunity to update to the latest cibuildwheel
version.
* make compatible with pylint and pycodestyle
* black config
* flake8 compatible config
* Missing newline
* Add black check to CI and dev requirements
This commit updates the CI configuration and local tox configuration to
leverage black instead of pycodestyle. It adds a check job to ci and the
tox lint job so we can quickly check if black has been run. A tox job
named 'black' is added to run black on all the code in the repo.
* Run black on everything
This commit reformats all the code in the qiskit-terra repository to use
black. It changes no functionality in the project and just adjusts the
code formatting to be consistent and automated. If you are looking at
this commit in the git log you can likely safely ignore any diff from
this commit as it is just the result of running '`black' on the repo
and instead you should look at any commits before or after this for
functional changes.
Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>
* Add macos universal2 wheel job
This commit adds a new wheel job for building universal2 wheels for
macOS. The universal2 wheels will enable running on arm64 macOS systems.
As we don't have any arm64 mac hardware available in CI systems yet
these jobs are cross-compiling arm64 binaries from an x86_64 mac, we
also can't test the arm64 binary component as part of the job because of
this.
This commit also takes the opportunity to update the cibuildwheel
version used for CI to the latest release 1.10.0, since we need to use
>=1.9.0 for arm on macOS support.
* DNM: Test wheel biulds
* Fix job name
* Revert "DNM: Test wheel biulds"
The wheel builds all worked and @nonhermitian manually tested the
universal2 binary on an arm64 mac. So this commit removes the test
commit to force the wheel jobs from running on the PR so this is ready
to be merged.
This reverts commit a9bafddcb8.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
With the pending support being added to qiskit-bot in
Qiskit/qiskit-bot#9 we're able to rename the default branch for the
repository to 'main'. However, when we do that several things will need
to be updated, most importantly the CI trigger was hardcoded to the
previous default branch 'master. This commit fixes these references and
should be merged after we rename the branch to re-enable CI.
* Cleanup stestr caching to only what's necessary
stestr maintains a local repository of run history which is used for
both local debugging (like for automated bisecting of failures between
runs) and also to provide data for parallel test scheduling based on the
the previous run time. In #5451 we added a cache step to the azure ci
configuration to cache the .stestr directory (the default location of
the stestr local repository) to preserve the run history to improve
parallel test scheduling to try and improve throughput. This however had
the unintended side effect of growing the stestr repository directory
significantly. Right now the stestr cache is several gigabytes in size.
This is because stestr will keep a history of all the runs in that
directory and by preserving it in a cache it's growing every time we
commit to the branch. However since we're only concerned with the timing
data we don't need the result streams for the runs to be preserved. This
commit adds a step to the CI jobs before we update the cache to remove
the results streams (which are just integer filenames in the repository
directory) and leave the dbm database of timing data. This will
significantly reduce the size of the data to cache, and also will limit
cache size growth to be a function of the number of tests in the suite
vs the number of runs in CI.
In a future stestr release there will be a command we can run to do this
for us (see mtreinish/stestr#306) but until that is released this commit
just manually deletes all the unecessary files prior to updating the
cache.
* Fix copy and paste error
* Workaround -d not existing in macOS xargs
* Workaround macOS grep not having -P
* Modify linux 3.7 CI job to install from sdist
Previously we were never explicitly testing the generation of an sdist
and whether we could build a package from that generated sdist. This
caught us by surprise in the 0.17.0 release because of an oversight
in #5941 we forgot to update the MANIFEST.in to include the new cython
source files which resulted in an sdist that could not be built. To
prevent potential issues like this in the future this commit modifies
one of the existing test jobs to explicitly generate an sdist and
install from that sdist instead of just pip installing the repo. This
will prevent such issues in the future.
* Try running in a different directory
* Make test suite self contained
When modifying a test job to build and install from a generated sdist
and run the unit tests from a non-repo directory a set of tests began to
fail because they are written with the implict assumption that tests are
always run from the git repo. This is primarily because the base test
class lives in the qiskit namespace which gets installed on install. In
the base test class there is a method used to find the path to resources
used for tests but that is written relative to the base class file which
at install time is not guaranteed to live in the same relative path to
the tests as when they're in the source repo. Since several other qiskit
projects rely on that base test class this commit just updates all the
tests which were relying on this method and have them directly use a
relative path from the test file so that all the tests work regardless
of how you install qiskit.
* Fix junitxml reporting
* Fix schemas test paths
* Fix typo
* Add example schemas to sdist too
* Fix lint
* Update test/python/test_qasm_parser.py
Co-authored-by: Lev Bishop <18673315+levbishop@users.noreply.github.com>
Co-authored-by: Lev Bishop <18673315+levbishop@users.noreply.github.com>
* Add lightweight v2 provider interface starter
This commit is a lighterweight v2 provider interface. This is an
alternative to what is built in #4885. While the model in #4885 is a
desireable end state it requires a lot of changes all at once for
providers and potentially users. Instead this commit brings the core
concept from #4885 of a cleaner explicitly versioned abstract interface
but minimizes the changes to the data model used in v1. Only some small
changes are made, mainly that jobs can be sync or async, Backend.run()
takes a circuit or schedule, options are set via an Options object at
__init__, with set_option(), or as kwargs on run(). In all other places
the object models from the v1 provider interface are used. This makes
the migration to a versioned interface simpler to start. From there we
can continue to evolve the interface as was done in #4485, like moving
to a target object, reimplementing properties and defaults as versioned
objects, etc.
Since the differences here are so small this commit brings the basicaer
provider over to the v2 provider interface. This can be done with
minimal effort to showcase how similar the 2 interfaces are.
* Fix basicaer simulator init
* Fix lint
* Add provider property to basicaer for Aqua backwards compat
* Add provider method back to backend class for backwards compat
* Fix lint
* Add release notes
* Add v2 provider to docs
* Fix lint
* Revert basicaer v2 provider migration
* Apply suggestions from code review
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>
* Add missing version attributes
* Make Options a simplenamespace subclass
* Update Backend docstrings
* Add v2 Backend support to the rest of terra
* Fix lint
* Fix lint
* Flatten providers subpackage
* Apply suggestions from code review
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Update release notes
* Migrate basic aer provider to v2 interface
This commit migrates the basic aer provider to the v2 interface. This
was originally included in #5086 but had to be removed because of a
potential backwards compatibility issue with aqua when using basic aer
as the provider (aqua 0.7.x explicity checks for v1 interface backends).
* DNM install aqua from source to test tutorials
* Remove deprecated schema validation
* Test failures
* Fix tests and lint
* Install aqua from source until release
* Add release notes
* Add ignis from source too as a dependency of aqua
* Apply suggestions from code review
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
* Finish upgrade release note
Co-authored-by: Ali Javadi-Abhari <ajavadia@users.noreply.github.com>
Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>
Jake Lishman