February 8, 2017 Hans-Peter Martini h2p3m

Simplifying Debugging of UI Tests

Selenium is a great tool for running UI based tests. You can get automated browser tests up and running in a matter of hours. However, as easy as Selenium is to set up. Selenium tests can be unstable, sometimes they work and now and then they fail.

Understanding why a Selenium test failed can often be a painful and time-consuming process. We have created this feature to take the pain out, and drastically reduce the time you spend to figure the causes out.

Smart Display Recordings

For all of your test commands in the tests section of your .scrutinizer.yml, we record any activity that occurs on the build machine’s display. If your test fails, the recording is stored and you can use it for quickly debugging the failure. Let’s take a look:

https://d2hs8c246tsqgl.cloudfront.net/blog/simple-debugging-of-ui-tests/65b71e0d11e9afe7021c74d3f7b3c4db-1b.gif

The display recording and the console output of the running task are displayed in sync. As the video is played, we re-play the output as it happened when your tests were running. So when you see an error on the display, you can see which step was executing on the command line and vice-versa.

Recordings are automatically enabled for all repositories, and even if you heavily use the UI, they cause virtually no overhead. We observed a maximum of 3% added CPU usage. If your task succeeds, the recording will be discarded. If it fails, we encode it at the end of your run so that you can view it in your browser.

Live Display Stream

While an inspection is running in SSH debug mode, you can now also view a live stream of the display without having to go through any complicated set-up directly in the browser. Checkout the new “Display” tab:

https://d2hs8c246tsqgl.cloudfront.net/blog/simple-debugging-of-ui-tests/73466d029c7ecd280b9df67b3724e414-2a.png

We think this will make Selenium tests a lot more enjoyable.

Let us know what you think! :)