Now that the first release candidate for the 0.24.0 release has been
tagged, the stable branch for the 0.24 series has been created and we
can start developing the 0.25.0 release on main. This commit bumps all
the version strings from 0.24.0 to 0.25.0 (and the backport branch for
mergify to 0.24) accordingly to differentiate the main branch from
0.24.*.
* Add structure for multiplie Rust extension crates
This is a precursor to adding an entirely separate and self-contained
crate to handle parsing of OpenQASM 2 (not 3 - this is kind of like a
trial run for that). It could conceivably still live within
`qiskit._accelerate`, but having separate crates helps us enforce more
sane API barriers, and has improvements in the incremental build time
when working on only one of the Rust extensions.
The intent after this commit is still to have `qiskit._accelerate` as an
easy catch-all for accelerators for Python. Developers should not need
to be concerned with defining a new crate for every new feature, and for
the most part `_accelerate` is still logically interoperative within
itself (for example, `NLayout` is used all over). This is just laying
out the groundwork so more complex crate additions _can_ also be made.
Some of the niceties to do with Cargo workspaces only became stabilised
in Rust 1.64. In particular, inheritance from the workspace root for
things like the package version, Rust version, and dependencies only
came in then. For now, the `workspace.packages` key in the root
`Cargo.toml` is ignored with a small warning during the build, but I've
put it in place mostly to keep track of what we can change once the MSRV
becomes 1.64 or greater (likely not til at least Terra 0.26).
* Add README.md to crates/accelerate
* Correct licence metadata
* Add `rust-toolchain.toml` for a consistent Rust development version
* Add components
* Simplify CI to not set Rust version
It will now use the rust-toolchain.toml file. This is possible because Rustup is already on the PATH, evidenced by us previously running `rustup default`
* Build Azure wheels with stable toolchain
* Use Stable for the sdist test build
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This formalises the MSRV policy by enforcing it at the configuration
level. This has helpful knock-on effects; `clippy` will no longer emit
linter failures for rules added in new versions of `clippy` that can't
be implemented in the MSRV. This makes our CI more stable against
updates to `clippy`.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit bumps the minimum supported rust version from 1.56.1 to
1.61.0 (which is the LCD for all our dependencies) which was released
on 2022-05-19. By doing this we're now able to bump all of our
upstream dependencies to use the latest releases.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Bump main branch version post 0.23.0rc1 tag
Now that the first release candidate for the 0.23.0 release has been
tagged, the stable branch for the 0.23 series has been created and we
can start developing the 0.24.0 release on main. This commit bumps all
the version strings from 0.23.0 to 0.24.0 (and the backport branch for
mergify to 0.23) accordingly to differentiate the main branch from
0.23.*.
* Bump version number in cargo lock file too
Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
The latest releases of PyO3 and rust numpy were just released. The
changelogs for both can be found at:
https://pyo3.rs/v0.18.0/changelog
and
https://github.com/PyO3/rust-numpy/blob/main/CHANGELOG.md
Since the two releases are tightly coupled this commit opts to update
them together instead of the normal dependabot workflow. Similarly some
of our usage (mainly around signatures and argument ordering) for
building a python interface using PyO3 has been deprecated. That usage
is also updated in this PR to avoid compilation warnings, which will be
treated as errors by clippy in CI.
* Improve performance of VF2 scoring and add support for scoring passes
This commit makes 2 key changes to the vf2 layout pass. The first is it
migrates the scoring routine to rust. When running vf2 layout and vf2
post layout we're bottlenecked by the performance of the scoring of a
layout since in practice scoring a large circuit ends up taking more
time than the vf2_mapping() function. To address this the scoring
function is migrated to rust where the iteration will be much faster. To
enable this rust migration the average error map is made into a 2D numpy
array which can be efficiently be accessed by reference from rust. This
additionally also enables a convenient interface for future expansion of
the vf2 layout passes. The VF2LayoutPass and VF2PostLayout passes will
now both look for a "vf2_avg_error_map" entry in the property set which
contains a 2d array used for scoring. If present that array will be used
for scoring instead of the computing one from the target's error rates.
This will enable custom analysis passes to be run pre-layout to compute
or inject a custom scoring heuristic.
* Handle missing qubits from properties Payload
For BackendV1 based backends it's possible for the BackendProperties
object for that beackend to get out of sync with the number fo qubits
actually available in the system. In such cases looking up the noise
characteristics can potentially fail when building the error map because
the reported number of qubits is less than the qubits there are
properties for. This wasn't an issue in the previous error map data
structure because it was a dictionary and it would just add the error
rate for the extra qubits even though it wasn't valid. However, now that
we're using a numpy array with a fixed size this isn't the case anymore
and an error would be raised in these cases. To workaround this issue
this commit skips any qubits outside the allowed range in the
BackendProperties when building the error map to account for this
potential discrepency. The extra properties couldn't be used anyway
since they're not valid device qubits in such cases.
* Limit number of intermediate Layout objects created
This commit updates the vf2 layout scoring to work with a dictionary
object instead of a Layout object. Previously we were creating a Layout
object on each mapping found and passing that to scoring. However, this
was unecessary overhead as the Layout object is slow to create and
interact with. Since we only need a Layout object if we're potentially
returning the layout as the best result we can avoid this extra
overhead.
* Move environment variable check outside loop
This commit removes the lookup for the QISKIT_IN_PARALLEL env variable
from the rust code for vf2 scoring. THis was adding unecessary overhead
to a frequently called function when it only needs to be computed once.
This commit moves the lookup to python outside the for loop and just
passes the evaluated boolean to the rust function instead.
* Fix rust lint
* Apply suggestions from code review
Co-authored-by: Jake Lishman <jake@binhbar.com>
* Simplify duplicated rust iteration code
This commit deduplicates a bunch of the rust side code for scoring into
2 closures and replaces all the reduce() calls with product() to
accomplish the same thing.
* Update qiskit/transpiler/passes/layout/vf2_layout.py
Co-authored-by: Jake Lishman <jake@binhbar.com>
* Use np.full() instead of np.empty() and np.fill()
* Pivot from 2d numpy array to a custom ErrorMap class
In order to support large (> 1000) qubit systems efficiently this commit
pivots away from using a 2d numpy array to represent the average error
rates for a target. For 1000q this error matrix would take 8 MB of
memory but for 10k qubit it would take 800 MB. Considering by their
nature these error matricies should be fairly sparse as connectivity in
typical QPUs is sparse. This was just wasted memory as we'll end up with
a lot of NaN values in the array. Instead this commit adds a new Rust
struct/Python class ErrorMap which just wraps a HashMap and maps a 2
element int array to a float. This way we only store entries where there
is defined connectivity and are more memory efficient.
* Fix lint
* Fix import path after rebase
* Update release notes
* Apply suggestions from code review
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Build empty ErrorMap in case of no target or coupling map
* Add helper function for layout creation in VF2Layout scoring loop
* Add test with custom ErrorMap analysis pass
Co-authored-by: Jake Lishman <jake@binhbar.com>
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Switch from retworkx to rustworkx package
In the retworkx/rustworkx 0.12.0 release the package was renamed from
retworkx to rustworkx which was done at the request of the networkx
maintainers [1]. While the retworkx name continues to work for backwards
compatibility, it is best to move off of the legacy name and start using
the new name sooner rather than later. This commit updates all the
retworkx usage to the new rustworkx and updates our requirements list to
directly depend on rustworkx.
[1] https://qiskit.org/documentation/rustworkx/release_notes.html#prelude
* Fix release note path
* Update releasenotes/notes/rustworkx-not-retworkx-b7c4da600df58701.yaml
Co-authored-by: Jake Lishman <jake@binhbar.com>
Co-authored-by: Jake Lishman <jake@binhbar.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Bump main branch version post rc tag
Now that the first release candidate for the 0.22.0 release has been
tagged, the stable branch for the 0.22 series has been created and we
can start developing the 0.23.0 release on main. This commit bumps all
the version strings from 0.22.0 to 0.23.0 (and the backport branch for
mergify to 0.22) accordingly to differentiate the main branch from
0.22.*.
* Move release notes to dedicated directory too
* Update Cargo too
Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
* Further oxidize sabre
In #7977 we started the process of oxidizing SabreSwap by replacing the
inner-most scoring heuristic loop with a rust routine. This greatly
improved the overall performance and scaling of the transpiler pass.
Continuing from where that started this commit migrates more of the pass
into the Rust domain so that almost all the pass's operations are done
inside a rust module and all that is returned is a list of swaps to run
prior to each 2q gate. This should further improve the runtime
performance of the pass and scaling because the only steps performed in
Python are generating the input data structures and then replaying the
circuit with SWAPs inserted at the appropriate points.
While we could have stuck with #7977 as the performance of the pass was
more than sufficient after it. What this commit really enables by moving
most of the pass to the rust domain is to expand with improvments and
expansion of the sabre algorithm which will require multithreaded to be
efficiently implemented. So while this will have some modest performance
improvements this is more about setting the stage for introducing
variants of SabreSwap that do more thorough analysis in the future
(which were previously preculded by the parallelism limitations of python).
* Fix most test failures
This commit fixes a small typo/logic error in the algorithm
implementation that was preventing sabre from making forward progress
because it wasn't correctly identifying successors for the next layer.
By fixing this all the hard errors in the SabreSwap tests are fixed. The
only failures left seem to be related to a different layout which
hopefully is not a correctness issue but just caused by different
ordering.
* Rework circuit reconstruction to use layer order
In some tests there were subtle differences in the relative positioning
of the 1q gates relative to inserted swaps (i.e. a 1q gate which was
before the swap previously could move to after it). This was caused by
different topological ordering being used between the hybrid python sabre
implementation and the mostly rust sabre implementations. To ensure a
consistent ordering fater moving mostly to rust this changes the
swap insertion loop to iterate over the circuit layers which mirrors how
the old sabre implementation worked.
* Differentiate between empty extended_set and none
* Simplify arguments passing to remove adjacency matrix storage
* Only check env variables once in rust
* Rust side NLayout.copy()
* Preserve SabreSwap execution order
This commit fixes an issue where in some cases the topological order the
DAGCircuit is traversed is different from the topological order that
sabre uses internally. The build_swap_map sabre swap function is only
valid if the 2q gates are replayed in the same exact order when
rebuilding the DAGCircuit. If a 2q gate gets replayed in a different
order the layout mapping will cause the circuit to diverge and
potentially be invalid. This commit updates the replay logic in the
python side to detect when the topological order over the dagcircuit
differs from the sabre traversal order and attempts to correct it.
* Rework SabreDAG to include full DAGCircuit structure
Previously we attempted to just have the rust component of sabre deal
solely with the 2q component of the input circuit. However, while this
works for ~80% of the cases it fails to account ordering and
interactions between non-2q gates or instructions with classical bits.
To address this the sabre dag structure is modified to contain all
isntructions in the input circuit and structurally match the
DAGCircuit's edges. This fixes most of the issues related to gate
ordering the previous implementation was encountering. It also
simplifies the swap insertion/replay of the circuit in the python side
as we now get an exact application order from the rust code.
* Switch back to topological_op_nodes() for SabreDAG creation
* Fix lint
* Fix extended set construction
* Fix typo in application of decay rate
* Remove unused QubitsDecay class
* Remove unused EdgeList class
* Remove unnecessary SabreRNG class
* Cleanup SabreDAG docstring and comments
* Remove unused edge weights from SabreDAG
The edge weights in the SabreDAG struct were set to the qubit indices
from the input DAGCircuit because the edges represent the flow of data
on the qubit. However, we never actually inspect the edge weights and
all having them present does is use extra memory. This commit changes
SabreDAG to just not set any weight for edges as all we need is the
source and target nodes for the algorithm to work.
* s/_bit_indices/_qubit_indices/g
* Fix sabre rust class signatures
* Reimplement SabreSwap heuristic scoring in multithreaded Rust
This commit re-implements the core heuristic scoring of swap candidates
in the SabreSwap pass as a multithread Rust routine. The heuristic
scoring in sabre previously looped over all potential swap candidates
serially in Python and applied a computed a heuristic score on which to
candidate to pick. This can easily be done in parallel as there is no
data dependency between scoring the different candidates. By performing
this in Rust not only is the scoring operation done more quickly for
each candidate but we can also leverage multithreading to do this
efficiently in parallel.
* Make sabre_swap a separate Rust module
This commit moves the sabre specific code into a separate rust module.
We already were using a separate Python module for the sabre code this
just mirrors that in the rust code for better organization.
* Fix lint
* Remove unnecessary parallel iteration
This commit removes an unecessary parallel iterator over the swap scores
to find the minimum and just does it serially. The threading overhead
for the parallel iterator is unecessary as it is fairly quick.
* Revert change to DECAY_RESET_INTERVAL behavior
* Avoid Bit._index
* Add __str__ definition for DEBUG logs
* Cleanup greedy swap path
* Preserve insertion order in SwapScores
The use of an inner hashmap meant the swap candidates were being
evaluated in a different order based on the hash seeding instead of the
order generated from the python side. This commit fixes by switching the
internal type to an IndexMap which for a little overhead preserves the
insertion order on iteration.
* Work with virtual indices win obtain swap
* Simplify decay reset() method
* Fix lint
* Fix typo
* Rename nlayout methods
* Update docstrings for SwapScores type
* Use correct swap method for _undo_operations()
* Fix rebase error
* Revert test change
* Reverse if condition in lookahead cost
* Fix missing len division on lookahead cost
* Remove unused EXTENDED_SET_WEIGHT python global
* Switch to serial iterator for heuristic scoring
While the heuristic scoring can be done in parallel as there is no data
dependency between computing the score for candidates the overhead of
dealing with multithreading eliminates and benefits from parallel
execution. This is because the relative computation is fairly quick and
the number of candidates is never very large (since coupling maps are
typically sparsely connected). This commit switches to a serial iterator
which will speed up execution in practice over running the iteration in
parallel.
* Return a 2d numpy array for best swaps and avoid conversion cost
* Migrate obtain_swaps to rust
This commit simplifies the rust loop by avoiding the need to have a
mutable shared swap scores between rust and python. Instead the obtain
swaps function to get the swap candidates for each layer is migrated to
rust using a new neighbors table which is computed once per sabre class.
This moves the iteration from obtain swaps to rust and eliminates it as
a bottleneck.
* Remove unused SwapScores class
* Fix module metadata path
* Add release note
* Add rust docstrings
* Pre-allocate candidate_swaps
* Double swap instead of clone
* Remove unnecessary list comprehensions
* Move random choice into rust
After rewriting the heuristic scoring in rust the biggest bottleneck in
the function (outside of computing the extended set and applying gates
to the dag) was performing the random choice between the best candidates
via numpy. This wasn't necessary since we can just do the random choice
in rust and have it return the best candidate. This commit adds a new
class to represent a shared rng that is reused on each scoring call and
changes sabre_score_heuristic to return the best swap.
The tradeoff with this PR is that it changes the seeding so when
compared to previous versions of SabreSwap different results will be
returned with the same seed value.
* Use int32 for max default rng seed for windows compat
* Fix bounds check on custom sequence type's __getitem__
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Only run parallel sort if not in a parallel context
This commit updates the sort step in the sabre algorithm to only run a
parallel sort if we're not already in a parallel context. This is to
prevent a potential over dispatch of work if we're trying to use
multiple threads from multiple processes. At the same time the sort
algorithm used is switched to the unstable variant because a stable sort
isn't necessary for this application and an unstable sort has less
overhead.
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Bump version strings post pre-release
Now that qiskit-terra 0.21.0rc0 is out the door we should bump the version
string on main to show the source version we're installing is newer
than what's been pre-released.
* Move 0.21 release notes into a separate directory
* Update cargo.toml too
* Refactor marginal_memory() hex to bin lookup table to be a true static
In the recently merged #8051 we create a lookup table in Rust to speed
up the hex->bin conversion used internally as part of the
marginal_memory() function. This was previously done using the
lazy_static crate which is used to lazily evaluate dynamic code to
create a static at runtime on the first access. The typical use case for
this is to create a static Vec or HashMap. However for the
marginal_counts() usage we didn't need to do this because we were
creating a fixed size array so the static can be evaulated at compile
time assuming the array is constructed with a const function. This
commit removes the lazy_static usage and switches to a true static to
further improve the performance of the lookup table by avoiding the
construction overhead.
* Reduce number of empty entries in LUT
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add dedicated functions for memory marginalization
This commit adds dedicated functions for memory marginalization.
Previously, the marginal_counts() function had support for marginalizing
memory in a Results object, but this can be inefficient especially if
your memory list is outside a Results object. The new functions added in
this commit are implemented in Rust and multithreaded. Additionally the
marginal_counts() function is updated to use the same inner Rust
functions.
* Fix rustfmt
* Add missing test file
* Fix typos
Co-authored-by: Daniel J. Egger <38065505+eggerdj@users.noreply.github.com>
* Remove unused import
* Increate default parallel_threshold to 1000
* Add support for different measurement levels
* Update docstring
* Add release note
* Expand unit tests
* Fix rustfmt
* Apply suggestions from code review
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Use a lookup table instead of a match statement
Co-authored-by: Daniel J. Egger <38065505+eggerdj@users.noreply.github.com>
Co-authored-by: Kevin Hartman <kevin@hart.mn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Set version number to 0.21.0rc1 for first release candidate
For the 0.21.0 release we're going to start pushing release candidates
prior to the release to enable testing before we cut the final release.
In preparation for tagging the first release candidate this commit
updates the version string to indicate it's a release candidate.
* Update cargo version to reflect 0.21.0
* Fix qpy handling of prerelease tags
* Fix pre-release handling in qpy compat tests
* Inline version parsing in qpy compat test
* Fix qpy version handling
* Reimplement DenseLayout as a Parallel Algorithm in Rust
This commit reimplements the core of the DenseLayout transpiler pass
in Rust and to run in multiple threads. Previously this algorithm used
scipy sparse matrices to create a CSR sparse matrix representation of
the coupling graph and iterate over that to find a densely connected
subgraph in the coupling graph. This performs and scales well for modest
sized circuits and coupling graphs. But as the size of the coupling
graphs and circuits grows running the algorithm by iterating over the
sparse matrix in Python starts to hit a limit. The underlying traversal
can be efficiently executed in parallel using rust as the algorithm
iterates over the coupling graph in BFS order for each node to try and
find the best subgraph. We can do the BFS traversals in parallel and
then iteratively compare the results in parallel until the best is
found. This greatly speeds up the execution of the pass, for example
running on a 1081 qubit quantum volume circuit on 1081 qubit
heavy hexagon coupling graph takes ~134 seconds with the previous
iteration and ~0.144 seconds after this commit (on my local workstation
with 32 physical cores and 64 logical cores, scaling likely won't be as
good on smaller systems).
The tradeoff here comes in slightly increased memory consumption as
to have a shared representation of the adjacency matrix (and the error)
between Python and Rust we use numpy arrays as they can be passed by
reference between the languages. In practice this will not matter much
until the graphs get truly large (e.g. to represent a 10000 qubit adjacency
matrix and error matrix would require ~1.6 GB of memory) and if it does
become an issue (either for memory or runtime performance) we can add a
shared compressed sparse matrix representation to Qiskit for use in both
Python in Rust.
* Fix measurement error calculation
* Fix lint
* Compute adjacency matrix at init instead of run()
* Fix test failures
This commit fixes the 4 remaining test failures. The results from the
rust version of the pass were correct but different than the results
from the Python version. This is because the parallel reduce() was
comparing in a different order that was returning a different subgraph.
This commit reverses the arg order to correct this so the behavior
should be identical to the previous implementation
* Fix error and matrix building
The error matrix building was not working because it was comparing a
list of qubits to a tuple. This was used prior to the rust rewrite so we
probably were not actually checking noise properties prior to this
commit.
* Add release notes
* Add rust docstring
* Update tests for fixed noise awareness
In an earlier commit we fixed the noise awareness of the laoyout pass,
doing this had a side effect of changing a test that was looking
explicitly for the layout found by the pass. Since the pass is now
correctly using error rates the best layout found is now different. This
commit updates the tests to account for this.
* Fix dag qubit count to include registerless bits
* Update docs and fix release note typos
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Reduce overhead of subgraph creation
This commit reduces the overhead of the internal loop over the bfs
sorted nodes to create the subgraph by making 2 changes. First, instead
of passing around the full bfs sorted list everywhere this commit
truncates it to just the nodes we're going to use. We never use any
qubits in the bfs sort > num_qubits so we can just truncate the Vec and
not have to worry about limiting to the first num_qubits elements. The
second change made is that instead of traversing the bfs nodes to check
if a node is in the subgraph we create an intermediate set to do a
constant time lookup for the membership check.
* Truncate bfs sort instead of truncating result
This commit adjusts the truncation logic around the bfs sort. In the
previous commit we truncated the bfs result to just the first n qubits
(where n is the number of qubits in the circuit) after performing a full
traversal and generating the full list. Instead of doing that we can
truncate the search when we've found n qubits already and save ourselves
the extra work of continuing to traverse the adjacency matrix.
* Simplify subgraph creation iterator using filter_map()
* Only run multithreaded outside of parallel context
This commit adds the env variable based switching to prevent us from
running dense layout in parallel when we're already running under
parallel_map to prevent overloading the system.
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Replace pauli expectation value cython with rust implementation
This commit replaces the cython implementation of the pauli expectation
value functions with a multithreaded rust implementation. This was done
primarily for two reasons, the first and primary reason for this change
is because after #7658 this module was the only cython code left in the
qiskit-terra repository so unifying on a single compiled language will
reduce the maintanence burden in qiskit-terra. The second reason is
similar to the rationale in #7658 around why using rust over cython for
multi-threaded hybrid python module. The difference here though is
unlike in stochastic swap this module isn't as performance critical as
it's not nearly as widely used.
* Tune single threaded performance for rust sum
This commit tunes the sum for the single threaded path. Using the
iterator sum() method is very convienent but for the single threaded
path it doesn't create the most efficient output. This was causing a
regression in performance over the previous cython version. To address
that issue, this commit adds a new tuned function which does a chunked
sum which the compiler can handle better. It more closely models how
we'd do this with vectorized SIMD instructions. As a future step we can
look at using simdeez https://github.com/jackmott/simdeez
to further optimize this by doing runtime CPU feature detection and
leveraging SIMD instrinsics (we might want to look at using `fast_sum()`
in the multithreaded path if we do that too).
* Add release notes
* Fix lint
* Add docstring and signature to rust functions
* Define parallel threshold as a constant
* Add attribution comment to fast_sum()
* Rename eval_parallel_env -> getenv_use_multiple_threads
* Use inline literal type for size
Co-authored-by: Kevin Hartman <kevin@hart.mn>
* Add overflow check on num_qubits
The functions only work for at most for number of qubits < usize bits
anything larger would cause an overflow. While rust provides overflow
checking in debug mode it disables this for performance in release mode.
Sice we ship binaries in release mode this commit adds an overflow check
for the num_qubits argument to ensure that we don't overflow and produce
incorrect results.
* Remove unecessary setup_requires field from setup.py
The setup_requires field in the setup.py is deprecated and
has been superseded by the pyproject.toml to define build
system dependencies. Since we're already relying on the
pyproject.toml to install setuptools-rust for us having the
setup_requires line will do nothing but potentially cause
issues as it will use an older install mechanism that will potentially conflict with people's environments.
* Drop `.iter().take(LANES)`.
* Fix typo.
Co-authored-by: Kevin Hartman <kevin@hart.mn>
Hashbrown 0.12.0 bumped the MSRV (minimum support rust version) to
1.56.1 which was released on November 2021. This is causing some issues
for people building locally as that is a pretty new version of rust.
While we don't set an MSRV for terra, I think we should try to at least
support that past ~6 months of Rust releases for building just to give
people time to upgrade and not require everyone to use rustup.
As there was nothing critical in hashbrown 0.12.0 this commit just
reverts back to 0.11.2 for now. We can look at bumping that after the
0.20.0 release.
This reverts commit d05931dbfa.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Implement multithreaded stochastic swap in rust
This commit is a rewrite of the core swap trials functionality in the
StochasticSwap transpiler pass. Previously this core routine was written
using Cython (see #1789) which had great performance, but that
implementation was single threaded. The core of the stochastic swap
algorithm by it's nature is well suited to be executed in parallel, it
attempts a number of random trials and then picks the best result
from all the trials and uses that for that layer. These trials can
easily be run in parallel as there is no data dependency between the
trials (there are shared inputs but read-only). As the algorithm
generally scales exponentially the speed up from running the trials in
parallel can offset this and improve the scaling of the pass. Running
the pass in parallel was previously tried in #4781 using Python
multiprocessing but the overhead of launching an additional process and
serializing the input arrays for each trial was significantly larger
than the speed gains. To run the algorithm efficiently in parallel
multithreading is needed to leverage shared memory on shared inputs.
This commit rewrites the cython routine using rust. This was done for
two reasons. The first is that rust's safety guarantees make dealing
with and writing parallel code much easier and safer. It's also
multiplatform because the rust language supports native threading
primatives in language. The second is while writing parallel cython
code using open-mp there are limitations with it, mainly on windows. In
practice it was also difficult to write and maintain parallel cython
code as it has very strict requirements on python and c code
interactions. It was much faster and easier to port it to rust and the
performance for each iteration (outside of parallelism) is the same (in
some cases marginally faster) in rust. The implementation here reuses
the data structures that the previous cython implementation introduced
(mainly flattening all the terra objects into 1d or 2d numpy arrays for
efficient access from C).
The speedups from this PR can be significant, calling transpile() on a
400 qubit (with a depth of 10) QV model circuit targetting a 409 heavy
hex coupling map goes from ~200 seconds with the single threaded cython
to ~60 seconds with this PR locally on a 32 core system, When transpiling
a 1000 qubit (also with a depth of 10) QV model circuit targetting a 1081
qubit heavy hex coupling map goes from taking ~6500 seconds to ~720
seconds.
The tradeoff with this PR is for local qiskit-terra development a rust
compiler needs to be installed. This is made trivial using rustup
(https://rustup.rs/), but it is an additional burden and one that we
might not want to make. If so we can look at turning this PR into a
separate repository/package that qiskit-terra can depend on. The
tradeoff here is that we'll be adding friction to the api boundary
between the pass and the core swap trials interface. But, it does ease
the dependency on development for qiskit-terra.
* Sanitize packaging to support future modules
This commit fixes how we package the compiled rust module in
qiskit-terra. As a single rust project only gives us a single compiled
binary output we can't use the same scheme we did previously with cython
with a separate dynamic lib file for each module. This shifts us to
making the rust code build a `qiskit._accelerate` module and in that we
have submodules for everything we need from compiled code. For this PR
there is only one submodule, `stochastic_swap`, so for example the
parallel swap_trials routine can be imported from
`qiskit._accelerate.stochastic_swap.swap_trials`. In the future we can
have additional submodules for other pieces of compiled code in qiskit.
For example, the likely next candidate is the pauli expectation value
cython module, which we'll likely port to rust and also make parallel
(for sufficiently large number of qubits). In that case we'd add a new
submodule for that functionality.
* Adjust random normal distribution to use correct mean
This commit corrects the use of the normal distribution to have the mean
set to 1.0. Previously we were doing this out of band for each value by
adding 1 to the random value which wasn't necessary because we could
just generate it with a mean of 1.0.
* Remove unecessary extra scope from locked read
This commit removes an unecessary extra scope around the locked read for
where we store the best solution. The scope was previously there to
release the lock after we check if there is a solution or not. However
this wasn't actually needed as we can just do the check inline and the
lock will release after the condition block.
* Remove unecessary explicit type from opt_edges variable
* Fix indices typo in NLayout constructor
Co-authored-by: Jake Lishman <jake@binhbar.com>
* Remove explicit lifetime annotation from swap_trials
Previously the swap_trials() function had an explicit lifetime
annotation `'p` which wasn't necessary because the compiler can
determine this on it's own. Normally when dealing with numpy views and a
Python object (i.e. a GIL handle) we need a lifetime annotation to tell
the rust compiler the numpy view and the python gil handle will have the
same lifetime. But since swap_trials doesn't take a gil handle and
operates purely in rust we don't need this lifetime and the rust
compiler can deal with the lifetime of the numpy views on their own.
* Use sum() instead of fold()
* Fix lint and add rust style and lint checks to CI
This commit fixes the python lint failures and also updates the ci
configuration for the lint job to also run rust's style and lint
enforcement.
* Fix returned layout mapping from NLayout
This commit fixes the output list from the `layout_mapping()`
method of `NLayout`. Previously, it incorrectly would return the
wrong indices it should be a list of virtual -> physical to
qubit pairs. This commit corrects this error
Co-authored-by: georgios-ts <45130028+georgios-ts@users.noreply.github.com>
* Tweak tox configuration to try and reliably build rust extension
* Make swap_trials parallelization configurable
This commit makes the parallelization of the swap_trials() configurable.
This is dones in two ways, first a new argument parallel_threshold is
added which takes an optional int which is the number of qubits to
switch between a parallel and serial version. The second is that it
takes into account the the state of the QISKIT_IN_PARALLEL environment
variable. This variable is set to TRUE by parallel_map() when we're
running in a multiprocessing context. In those cases also running
stochastic swap in parallel will likely just cause too much load as
we're potentially oversubscribing work to the number of available CPUs.
So, if QISKIT_IN_PARALLEL is set to True we run swap_trials serially.
* Revert "Make swap_trials parallelization configurable"
This reverts commit 57790c84b0. That
commit attempted to sovle some issues in test running, mainly around
multiple parallel dispatch causing exceess load. But in practice it was
broken and caused more issues than it fixed. We'll investigate and add
control for the parallelization in a future commit separately after all
the tests are passing so we have a good baseline.
* Add docs to swap_trials() and remove unecessary num_gates arg
* Fix race condition leading to non-deterministic behavior
Previously, in the case of circuits that had multiple best possible
depth == 1 solutions for a layer, there was a race condition in the fast
exit path between the threads which could lead to a non-deterministic
result even with a fixed seed. The output was always valid, but which
result was dependent on which parallel thread with an ideal solution
finished last and wrote to the locked best result last. This was causing
weird non-deterministic test failures for some tests because of #1794 as
the exact match result would change between runs. This could be a bigger
issue because user expectations are that with a fixed seed set on the
transpiler that the output circuit will be deterministically
reproducible.
To address this is issue this commit trades off some performance to
ensure we're always returning a deterministic result in this case. This
is accomplished by updating/checking if a depth==1 solution has been
found in another trial thread we only act (so either exit early or
update the already found depth == 1 solution) if that solution already
found has a trial number that is less than this thread's trial number.
This does limit the effectiveness of the fast exit, but in practice it
should hopefully not effect the speed too much.
As part of this commit some tests are updated because the new
deterministic behavior is slightly different from the previous results
from the cython serial implementation. I manually verified that the
new output circuits are still valid (it also looks like the quality
of the results in some of those cases improved, but this is strictly
anecdotal and shouldn't be taken as a general trend with this PR).
* Apply suggestions from code review
Co-authored-by: georgios-ts <45130028+georgios-ts@users.noreply.github.com>
* Fix compiler errors in previous commit
* Revert accidental commit of parallel reduction in compute_cost
This was only a for local testing to prove it was a bad idea and was
accidently included in the branch. We should not nest the parallel
execution like this.
* Eliminate short circuit for depth == 1 swap_trial() result
This commit eliminates the short circuit fast return in swap_trial()
when another trial thread has found an ideal solution. Trying to do this
in a parallel context is tricky to make deterministic because in cases
of >1 depth == 1 solutions there is an inherent race condition between
the threads for writing out their depth == 1 result to the shared
location. Different strategies were tried to make this reliably
deterministic but there wa still a race condition. Since this was just a
performance optimization to avoid doing unnecessary work this commit
removes this step. Weighing improved performance against repeatability
in the output of the compiler, the reproducible results are more
important. After we've adopted a multithreaded stochastic swap we can
investigate adding this back as a potential future optimization.
* Add missing docstrings
* Add section to contributing on installing form source
* Make rust python classes pickleable
* Add rust compiler install to linux wheel jobs
* Try more tox changes to fix docs builds
* Revert "Eliminate short circuit for depth == 1 swap_trial() result"
This reverts commit c510764a77. The
removal there was premature and we had a fix for the non-determinism in
place, ignoring a typo which was preventing it from working.
Co-Authored-By: Georgios Tsilimigkounakis <45130028+georgios-ts@users.noreply.github.com>
* Fix submodule declaration and module attribute on rust classes
* Fix rust lint
* Fix docs job definition
* Disable multiprocessing parallelism in unit tests
This commit disables the multiprocessing based parallelism when running
unittest jobs in CI. We historically have defaulted the use of
multiprocessing in environments only where the "fork" start method is
available because this has the best performance and has no caveats
around how it is used by users (you don't need an
`if __name__ == "__main__"` guard). However, the use of the "fork"
method isn't always 100% reliable (see
https://bugs.python.org/issue40379), which we saw on Python 3.9 #6188.
In unittest CI (and tox) by default we use stestr which spawns (not using
fork) parallel workers to run tests in parallel. With this PR this means
in unittest we're now running multiple test runner subprocesses, which
are executing parallel dispatched code using multiprocessing's fork
start method, which is executing multithreaded rust code. This three layers
of nesting is fairly reliably hanging as Python's fork doesn't seem to
be able to handle this many layers of nested parallelism. There are 2
ways I've been able to fix this, the first is to change the start method
used by `parallel_map()` to either "spawn" or "forkserver" either of
these does not suffer from random hanging. However, doing this in the
unittest context causes significant overhead and slows down test
executing significantly. The other is to just disable the
multiprocessing which fixes the hanging and doesn't impact runtime
performance signifcantly (and might actually help in CI so we're not
oversubscribing the limited resources.
As I have not been able to reproduce `parallel_map()` hanging in
a standalone context with multithreaded stochastic swap this commit opts
for just disabling multiprocessing in CI and documenting the known issue
in the release notes as this is the simpler solution. It's unlikely that
users will nest parallel processes as it typically hurts performance
(and parallel_map() actively guards against it), we only did it in
testing previously because the tests which relied on it were a small
portion of the test suite (roughly 65 tests) and typically did not have
a significant impact on the total throughput of the test suite.
* Fix typo in azure pipelines config
* Remove unecessary extension compilation for image tests
* Add test script to explicitly verify parallel dispatch
In an earlier commit we disabled the use of parallel dispatch in
parallel_map() to avoid a bug in cpython associated with their fork()
based subprocess launch. Doing this works around the bug which was
reliably triggered by running multiprocessing in parallel subprocesses.
It also has the side benefit of providing a ~2x speed up for test suite
execution in CI. However, this meant we lost our test coverage in CI for
running parallel_map() with actual multiprocessing based parallel
dispatch. To ensure we don't inadvertandtly regress this code path
moving forward this commit adds a dedicated test script which runs a
simple transpilation in parallel and verifies that everything works as
expected with the default parallelism settings.
* Avoid multi-threading when run in a multiprocessing context
This commit adds a switch on running between a single threaded and a
multithreaded variant of the swap_trials loop based on whether the
QISKIT_IN_PARALLEL flag is set. If QISKIT_IN_PARALLEL is set to TRUE
this means the `parallel_map()` function is running in the outer python
context and we're running in multiprocessing already. This means we do
not want to be running in multiple threads generally as that will lead
to potential resource exhaustion by spawn n processes each potentially
running with m threads where `n` is `min(num_phys_cpus, num_tasks)` and
`m` is num_logical_cpus (although only
`min(num_logical_cpus, num_trials)` will be active) which on the typical
system there aren't enough cores to leverage both multiprocessing and
multithreading. However, in case a user does have such an environment
they can set the `QISKIT_FORCE_THREADS` env variable to `TRUE` which
will use threading regardless of the status of `QISKIT_IN_PARALLEL`.
* Apply suggestions from code review
Co-authored-by: Jake Lishman <jake@binhbar.com>
* Minor fixes from review comments
This commits fixes some minor details found during code review. It
expands the section on building from source to explain how to build a
release optimized binary with editable mode, makes the QISKIT_PARALLEL
env variable usage consistent across all jobs, and adds a missing
shebang to the `install_rush.sh` script which is used to install rust in
the manylinux container environment.
* Simplify tox configuration
In earlier commits the tox configuration was changed to try and fix the
docs CI job by going to great effort to try and enforce that
setuptools-rust was installed in all situations, even before it was
actually needed. However, the problem with the docs ci job was unrelated
to the tox configuration and this reverts the configuration to something
that works with more versions of tox and setuptools-rust.
* Add missing pieces of cargo configuration
Co-authored-by: Jake Lishman <jake@binhbar.com>
Co-authored-by: georgios-ts <45130028+georgios-ts@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Matthew Treinish