-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Metatest for swap mappers #1571
Conversation
…erra into metatest_mapper
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the changes, @1ucian0. I think we are almost there. How do you plan to remove the old pickles in case the name of the circuit changes for some reason?
Hey, @1ucian0, what about removing the pickles that are no longer needed after regenerating? |
done in 0d65f25 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great. My only concern is when to regenerate, I know there are plans for including a developer guide about the transpiler and passes and we can include information about it there (#83). Another option is to add some information to the CONTRIBUTING.rst
guidelines, in the same way we did when we updated the test options (#1832).
* Issue Qiskit#1186: Add test file for circuit drawers This commit adds a new file `test_circuit_drawer`. It is created to test all the circuit drawers: `circuit_drawer` as well as the underlying circuit drawers. This very commit contains a test case to test only behavior of `circuit_drawer`. In particular, it checks how `circuit_drawer` handles different provided values of `output` parameter. * Add line breaks to follow the codestyle * Add test case for underlying circuit drawers This commit adds a test case for underlying circuit drawers. At the moment, it tests only how circuit drawers behave while drawing a small circuit. However, further tests of behaviour on greater circuits will be added. The test case works as follows: there are presaved references of how each circuit drawer shall draw a particular small circuit which is obtained upon calling of `_small_circuit` funtion. The test case makes all circuit drawer to produce their versions of the small circuit and then compares these versions with the corresponding references. Since latex and matplotlib drawers produce pictures, the test case compares the produced images while for text and latex source drawers it compares bytes of the files. * Add referencesfor medium circuit * Remove redundant variable * Issue Qiskit#1886: Improve tests of circuit drawers This commit completes the test case for different circuit drawers. In particular, it adds tests and references for all circuit types. Also it adds comments and structurizes the code. * Force to skip testing of deprecated circuit_drawer fallback * According to Qiskit#1628, import of QiskitTestCase was changed from relative to absolute one * Docstrings of functions were refactored in accordance with the project codestyle. In particular, format of return string was changed. * This commit changes defining the path to references locally in the test file in favor of defining it qiskit-wide and the using _get_resource_path of QiskitTestCase * This commit moves common part of tests into a single function. Thus, it significantly reduces code repetition. * Update medium and large references for mpl * As `assert_called_once` is available only for Python 3.6, the corresponding usages of mock asserts were updated so that they can be supported with Python 3.5. In particular, the were replaced to `assert_called_once_with`. * Fix style bug * Add readable assert fail messages * As different Python version (at least second digit matters, probably versions of packages matters too, it is unknown up to now), this commit adds mechanism to handle current Python version and to look for references in the corresponding folder. * Fix style bugs * There were still some `assert_called_once` left after previous commits, so I have now replaced them with `assert_called_once_with` and added a dict of correspinding calling arguments * Fix style bug * Up to this moment all registers had their names generated automatically. Hence, after consequent runs of several tests, their names could have changed randomly. There was a function `_reset_indices` to cope with this, but it didn't take into account any intermediate runs of other tests. Now, each register is assigned with a constant name and thus no fails should happen * This commit changes the name to be set for all created registers and changes the references correspondingly. In addition, it fixes a small style bug. * It seems that Travis has no pdflatex installed and thus, testing of LaTeX drawer will obviously fail. So, this commit add handling of an OSError. * Add references for Python 3.7 * During previous commits it was discovered that `text_drawer` produces different outputs for different versions of Python while drawing the `deep_circuit`. In particular, outputs for Python 3.5, and 3.6 with 3.7 differ. A notorious fact is that they are not different in sense of some minor things like spacing, they differs like there is a bug. So, until this issue is managed, there should be no `deep_circuit` testing. Consequently, since all other cases produce similar outputs, they can be evaluated similarly without paying attention to the dependence on version. * Thanks to @diego-plan9, this commits just replaces manual creation of temporary directory with standard Python function. * Update references for TestVisualizationImplementation * Thanks to @diego-plan9, this commit moves functions common for TestDrawingMethods and TestVisualizationImplementation to a common base class, DrawingTestCase. Consequently, these test cases are inherited from this class and still possess their previous functinalty. * Fix style bug * Fix style bugs * Fix style bugs * Fix style bug * Following the @1ucian0 comment, this commit removes test for deprecated fallback * Fix style errors * This commit at first, of course, update references to keep them in touch with the underlying repository updates. But, as well, it removes dependency on version of Python from the commented part of a setUp function * This commit follows the principle Qiskit#1571 and introduces an automatic regeneration of references as soon as test file is called as a standalone script. It is done as follows: as soon as this script is a main one, it calls a static method of TestDrawingMethods class, which regenerates the references, just before calling of the unittest facilities. In addition, this commit moves decalaration of all variables to the body of class, not to the setUp methods, which turned out to be redundant. Finally, it introduces a `setUpClass` for a TestDrawingMethods class to create a ONE temporary folder for ALL the tests, not for each one separately. * Fix style bug
Fixes #1461.
This PR is a second attempt to have a simple meta test for all the "swappers".
The test checks the output of the swapper to a ground truth DAG (one for each test/swapper) saved in as a pickle (in
test/python/pickles/
). If they need to be regenerated, the DAG candidate is compiled and run in a simulator and the count is checked before being saved.How to add a swapper
Add the following class in
test/python/transpiler/test_mappers.py
How to add a test for all the swappers
Add the following method in the class
CommonTestCases
: