The Sentry engineering team recently recounted the drivers and lessons learned from migrating its front-end tests from Enzyme to the React Testing Library on its blog. The migration was triggered by Enzyme’s lack of support for newer versions of React. The migration took about 20 months and involved 17 engineers reviewing around 5,000 tests.
Sentry’s engineering times decided several times against migrating their test base to React Testing Library (RTL) for lack of substantial-enough benefits. The team recalled:
We don’t just throw things in because they’re new. We carefully evaluate new technologies to understand what benefits they bring to our team. RTL was known to us back then, but we didn’t have strong arguments about why we should bring it into our codebase. Enzyme, the library we used to test our component library, was still attending to our needs.
On the one hand, Sentry was already engaged in a large migration to TypeScript, which, together with regular product work, was keeping the engineering team busy.
On the other hand, Enzyme tests often took very long, and the team had a strong interest in improving test speed.
(Source: Sentry’s engineering blog)
A proof of concept showed a 12% performance improvement, which was deemed insufficient to embark on yet another long migration project. The proof of concept nonetheless proved that RTL had observable advantages over Enzyme. As the team reports, Enzyme did not test for accessibility, did not automatically clean up the test environment, and often directly accessed the component under the test’s state. Conversely, RTL is closer to integration testing and strives to test application use cases from the user’s perspective. In particular, RTL strives to avoid testing implementation details. Implementation changes should only break a test if they indeed introduced a bug.
The tradeoff analysis changed after Sentry migrated to TypeScript and started to upgrade to React 17 (which includes React Hooks). The team recalls:
The [RTL] migration still didn’t get much attention until we worked on updating React to version 17. The React core team had completely rewritten the library’s internals and Enzyme directly used a number of internal React functionality.
[…] Enzyme didn’t work 100% with this new version of React, but there was an adapter on the market that worked around this problem and that’s what we used. However, this solution wouldn’t work in the long run as React 18 would require a complete rewrite, which was unlikely to happen given that Airbnb had dropped support for Enzyme.
[…] RTL does not rely on React’s internals. and would continue to work the same with React 18 as it did with 16 and 17.
Once the green light was given, the focus switched to minimizing the risks of the migration project (engineering estimates under various hypotheses, iterative approach vs big-bang migration, progress tracking, RTL training, surfacing of best practices, daily code reviews, and more).
The migration was completed after 18 months (vs an estimated 14 months). It allowed the team to remove obsolete tests, improve accessibility - a previously overlooked aspect, and write tests based on use cases instead of implementation details.
The team detailed unexpected performance issues encountered when following some RTL recommendations to the letter (e.g., in addition to mocking web APIs, mock the user as much and as realistically as possible). Despite not seeing dramatic improvements on the test performance front (the primary pain point that drove the interest for the initial proof of concept), the team positively concluded:
Although the performance of our tests has not improved as we had hoped, the introduction of the library has brought many other benefits, such as tests that no longer rely on implementation details, but instead test what the user sees and interacts with. And ultimately, that’s what matters.
In the original article, the Sentry team discusses interesting technical details accompanied by quantitative data and qualitative illustrations that may interest other engineering teams. Interested developers are invited to refer to it for further information.
React Testing Library is one of a suite of user-interface testing libraries (e.g. DOM Testing Library, Vue Testing Library, Svelte Testing Library, Puppeteer Testing Library) with a similar driving philosophy:
You want your test base to be maintainable in the long run so refactors of your components (changes to implementation but not functionality) don’t break your tests and slow you and your team down. The
DOM Testing Library[provides utilities to] query the DOM for nodes in a way that’s similar to how the user finds elements on the page […] The more your tests resemble the way your software is used, the more confidence they can give you.