Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage News Flutter Golden Tests Powering Automated UI Testing at eBay

Flutter Golden Tests Powering Automated UI Testing at eBay

This item in japanese

Automating UI tests to ensure the visual appearance of your app is correct on all supported devices and form factors is usually challenging. Flutter golden tests may help simplify this task, as eBay's approach to UI screenshot testing for its Motors app shows.

Visual UI testing, as opposed to behavioural UI testing, can easily become a cumbersome and expensive process, especially with rich client applications:

How the UI looks on a single device is only one consideration. From text size to accessibility to different languages, engineers must think about – and test for – a seemingly endless number of possibilities.

Besides the sheer number of combinations of devices, form factors, languages, and device orientations, what makes visual UI testing specifically hard is it is not easy to say what is correct and what not. This is how Gojko Adzic, creator of visual testing tool Appraise, explains why it is so:

With visuals, it's often difficult to express correctness. Someone can design lots of specific checks for, say, element size, positioning and styling, but the whole composition on the screen might still be wrong, weird, or just ugly. Though people can easily spot when something is off, because it's difficult to express expected results, automation traditionally wasn’t very helpful.

As an effect of this, testing an app's visuals requires some humans review the final result visually, which can in turn slow down the whole development process and the creation of new features, says Adzic.

One approach to this problem is screenshot testing, which consists of taking a number of screenshots of your app and then comparing them with reference screenshots. Any deviation between any screenshot pairs will be flagged for further review.

This process is simplified by Flutter Gold. This is an automation tool that enables the rendering and capture of a Flutter widget as a screenshot, the storage of this as a baseline in a dedicated repo, and a process to compare each reference screenshot to the one produced at build time after a change.

eBay evolved its use of Flutter Gold to move beyond simple widget testing. It now supports testing of widgets with multiple layout variations and testing of a composition of multiple widgets, as exemplified in the following image:

In the process of using the tool, eBay found that Flutter Gold required redundant configuration code to be written into new tests. This compounded with the complexity of managing a large number of UI tests. To tackle this problem, eBay engineers created a Flutter package, Golden Toolkit, to encapsulate a number of recurring patterns and simplify the problem space.

According to eBay, this approach has made it possible to move quickly without sacrificing on their app quality.

Rate this Article