Update the docs
This commit is contained in:
Michael Mintz
parent
4a814577bd
commit
50718e6fc9
|
|
@ -8,7 +8,7 @@
|
|||
<p align="center"><a href="https://github.com/seleniumbase/SeleniumBase/">
|
||||
<img src="https://seleniumbase.io/img/sb_logo_10.png" alt="SeleniumBase" title="SeleniumBase" width="280" /></a></p>
|
||||
<p align="center">
|
||||
<b>Everything you need to test websites.</b>
|
||||
<b>Build fast, reliable, end-to-end tests.</b>
|
||||
</p>
|
||||
|
||||
<!-- View on GitHub -->
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
<p align="center"><a href="https://github.com/seleniumbase/SeleniumBase/"><img src="https://seleniumbase.io/cdn/img/sb_logo_f6.png" alt="SeleniumBase" width="320" /></a></p>
|
||||
|
||||
## <img src="https://seleniumbase.io/img/sb_icon.png" title="SeleniumBase" width="30" /> Automated Visual Regression Testing
|
||||
<h2><img src="https://seleniumbase.io/img/sb_icon.png" title="SeleniumBase" width="30" /> Automated Visual Regression Testing</h2>
|
||||
|
||||
Automated Visual Regression Testing can help you detect when the layout of a web page has changed. Instead of comparing pixels from screenshots, layout differences can be detected by comparing HTML tags and attributes with a baseline. If a change is detected, it could mean that something broke, the web page was redesigned, or dynamic content changed.
|
||||
|
||||
|
|
@ -10,35 +10,42 @@ Automated Visual Regression Testing can help you detect when the layout of a web
|
|||
|
||||
To handle automated visual testing, SeleniumBase uses the ``self.check_window()`` method, which can set visual baselines for comparison and then compare the latest versions of web pages to the existing baseline.
|
||||
|
||||
The first time a test calls ``self.check_window()`` with a unique "name" parameter, the visual baseline is set, which means a folder is created with the following files:
|
||||
* page_url.txt -> The URL of the current window
|
||||
* screenshot.png -> A screenshot of the current window
|
||||
* tags_level1.txt -> HTML tags from the window
|
||||
* tags_level2.txt -> HTML tags + attribute names
|
||||
* tags_level3.txt -> HTML tags + attribute names+values
|
||||
The first time a test calls ``self.check_window()`` with a unique ``name`` parameter, the visual baseline is set, which means a folder is created with the following files:
|
||||
|
||||
<li><b>page_url.txt</b> -> The URL of the current window</li>
|
||||
<li><b>screenshot.png</b> -> A screenshot of the current window</li>
|
||||
<li><b>tags_level1.txt</b> -> HTML tags from the window</li>
|
||||
<li><b>tags_level2.txt</b> -> HTML tags + attribute names</li>
|
||||
<li><b>tags_level3.txt</b> -> HTML tags + attribute names+values</li>
|
||||
|
||||
After the first time ``self.check_window()`` is called, later calls will compare the HTML tags and attributes of the latest window to the ones from the first call (<i>or to the ones from the call when the baseline was last reset</i>).
|
||||
|
||||
Here's an example call:
|
||||
```
|
||||
|
||||
```python
|
||||
self.check_window(name="first_test)", level=3)
|
||||
```
|
||||
|
||||
On the first run (<i>or if the baseline is being set/reset</i>) the "level" doesn't matter because that's only used for comparing the current layout to the existing baseline.
|
||||
|
||||
Here's how the level system works:
|
||||
* level=0 ->
|
||||
DRY RUN ONLY - Will perform a comparison to the baseline, and print out any differences that are found, but won't fail the test even if differences exist.
|
||||
* level=1 ->
|
||||
HTML tags are compared to tags_level1.txt
|
||||
* level=2 ->
|
||||
HTML tags and attribute names are compared to tags_level2.txt
|
||||
* level=3 ->
|
||||
HTML tags and attribute names+values are compared to tags_level3.txt
|
||||
|
||||
<li><b>level=0</b> ->
|
||||
DRY RUN ONLY - Will perform a comparison to the baseline, and print out any differences that are found, but won't fail the test even if differences exist.</li>
|
||||
<li><b>level=1</b> ->
|
||||
HTML tags are compared to tags_level1.txt</li>
|
||||
<li><b>level=2</b> ->
|
||||
HTML tags and attribute names are compared to tags_level2.txt</li>
|
||||
<li><b>level=3</b> ->
|
||||
HTML tags and attribute names+values are compared to tags_level3.txt</li>
|
||||
|
||||
As shown, Level-3 is the most strict, Level-1 is the least strict. If the comparisons from the latest window to the existing baseline don't match, the current test will fail, except for Level-0 checks, which print Level-3 results without failing the test.
|
||||
|
||||
You can reset the visual baseline on the command line by adding the following parameter at runtime:
|
||||
``--visual_baseline``
|
||||
|
||||
```bash
|
||||
--visual_baseline
|
||||
```
|
||||
|
||||
As long as ``--visual_baseline`` is used on the command line while running tests, the ``self.check_window()`` method cannot fail because it will rebuild the visual baseline rather than comparing the html tags of the latest run to the existing baseline. If there are any expected layout changes to a website that you're testing, you'll need to reset the baseline to prevent unnecessary failures.
|
||||
|
||||
|
|
@ -49,6 +56,7 @@ If you want to use ``self.check_window()`` to compare a web page to a later vers
|
|||
Automated Visual Testing with ``self.check_window()`` is not very effective for websites that have dynamic content because that changes the layout and structure of web pages. For those pages, you're much better off using regular SeleniumBase functional testing, unless you can remove the dynamic content before performing the comparison, (such as by using ``self.ad_block()`` to remove dynamic ad content on a web page).
|
||||
|
||||
Example usage of ``self.check_window()`` with different levels:
|
||||
|
||||
```python
|
||||
self.check_window(name="testing", level=0)
|
||||
self.check_window(name="xkcd_home", level=1)
|
||||
|
|
@ -61,6 +69,7 @@ Example usage of ``self.check_window()`` with different levels:
|
|||
```
|
||||
|
||||
Here's an example where clicking a button makes a hidden element visible:
|
||||
|
||||
```python
|
||||
from seleniumbase import BaseCase
|
||||
|
||||
|
|
@ -76,7 +85,9 @@ class VisualLayoutTest(BaseCase):
|
|||
self.click("button")
|
||||
self.check_window(name="helloworld", level=3)
|
||||
```
|
||||
|
||||
Here's the output of that: (<i>Text changes do not impact visual comparisons</i>)
|
||||
|
||||
```
|
||||
AssertionError:
|
||||
First differing element 39:
|
||||
|
|
@ -92,6 +103,7 @@ First differing element 39:
|
|||
```
|
||||
|
||||
Here's an example where a button is removed from a web page:
|
||||
|
||||
```python
|
||||
from seleniumbase import BaseCase
|
||||
|
||||
|
|
@ -105,7 +117,9 @@ class VisualLayoutTest(BaseCase):
|
|||
self.remove_element('a.donate-button')
|
||||
self.check_window(name="python_home", level=3)
|
||||
```
|
||||
|
||||
Here's the output of that:
|
||||
|
||||
```
|
||||
AssertionError:
|
||||
First differing element 33:
|
||||
|
|
@ -123,6 +137,7 @@ First differing element 33:
|
|||
```
|
||||
|
||||
Here's an example where a web site logo is resized:
|
||||
|
||||
```python
|
||||
from seleniumbase import BaseCase
|
||||
|
||||
|
|
@ -137,7 +152,9 @@ class VisualLayoutTest(BaseCase):
|
|||
self.set_attribute('[alt="xkcd.com logo"]', "width", "120")
|
||||
self.check_window(name="xkcd_554", level=3)
|
||||
```
|
||||
|
||||
Here's the output of that:
|
||||
|
||||
```
|
||||
AssertionError:
|
||||
First differing element 22:
|
||||
|
|
@ -158,9 +175,11 @@ First differing element 22:
|
|||
```
|
||||
|
||||
To run the example (from [examples/visual_testing/](https://github.com/seleniumbase/SeleniumBase/blob/master/examples/visual_testing/)) with a pytest HTML Report, use:
|
||||
```
|
||||
|
||||
```bash
|
||||
pytest test_layout_fail.py --html=report.html
|
||||
```
|
||||
|
||||
Here's what the pytest HTML Report looks like:<br />
|
||||
[<img src="https://seleniumbase.io/cdn/img/visual_testing_report_2.png" title="Test Report">](https://seleniumbase.io/cdn/img/visual_testing_report_2.png)
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue