mirror of https://github.com/Qiskit/qiskit.git
1 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Matthew Treinish
|
6eb436e411 |
Switch to stestr for test runner (#737)
* Switch to stestr for test runner This commit switches the default test runner for qiskit-terra to use stestr instead of the built-in runner from stdlib unittest. By default this switches the default mode of execution to be parallel instead of serial which can lead to significant speed ups in test execution time. Also the junitxml upload on the appveyor ci is switched to use the subunit protocol instead of the xunit format. This is because subunit is used natively by stestr. While we can generate xunit xml in the same exact format it doesn't work on windows because of the subunit library bug: https://bugs.launchpad.net/subunit/+bug/1322888 . Once this bug is fixed we can convert the subunit to xunit xml as part of the CI job. That being said though, the subunit data contains every thing and whatever post processing is being done on the xml files can easily be converted to use the subunit. Closes #730 * Remove initial check for initializing state The initializing state for a job is a transient condition. Depending on how fast the test executes we may or may not be able to catch it during the execution of the test. When running the test suite in parallel we're seeing many instances where things execute too quickly and the assert looking for the initial initializing state is too slow and the api status has already moved on. This commit fixes this by simply removing that check from all the tests since it can't be reliably checked. * Stop writing progress bar to stdout during tests This commit adjust the parallel_progress bar unit tests to mock out sys.stdout.write() so we aren't actually writing the progress bar to stdout during the test run. We're not actually verifying the contents written to stdout anywhere in the tests and this both pollutes stdout during a run and also fails when running tests under stestr on windows ci, likely because of how python-subunit (a external dependency library of stestr's) internally handles stdout encapsulation. So to avoid these this commit mocks out the stdout writes and asserts we're at least calling it. In the future we can adjust these to actually verify the calls for the proper output. But for right now asserting we're even calling sys.stdout.write() is an improvement and avoids the issues. * Fix typos in tox.ini * Change to per test method parallelization by default Prior to this commit we were using test class level parallelization (ie each test class is the minimum unit that gets passed to a worker). This was done to optimize for the online tests which have expensive class level constructors which we wouldn't want to run multiple times if the individual test methods got split across multiple workers. However, this was hurting the overall performance of the more common case where we don't make actual api requests. This commit switches us to use per method parallelization by default and manually switch to per class for the online tests in CI. * Revert "Change to per test method parallelization by default" This reverts commit 0ab794b0a5bc14d23e1820042a70f66b0d68c99b. * Update appveyor comments for using test_results.xml This commit updates the comments in the appveyor to refer to the upstream fix needed to leverage the test_results.xml. It also adds all the necessary lines to properly generate an xunit xml file and upload it. |