Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage News Shifting Quality Left with the Test Pyramid

Shifting Quality Left with the Test Pyramid

This item in japanese

Shifting quality left means building in quality much earlier in the software development cycle, rather than testing for quality after completion of development. Using the test pyramid model, a project was able to move testing towards earlier stages, thereby finding defects that caused integration issues earlier in development.

Haritha Hari, a lead quality analyst at ThoughtWorks, Australia, spoke about shifting quality left at ConTEST 2021.

Embedding quality into the system means every code that is committed is production-ready, and thereby no additional effort is required in testing, Haritha argued.

The organisation faced integration issues where the systems did not work end-to-end after the completion of development. In order to prevent this delayed production deployment, integration issues had to be addressed at the beginning. This was possible only through shift-left testing wherein testing started right from the design phase, Haritha said.

To practically approach this principle of shift-left testing, they adopted the test pyramid model of testing strategy. According to the test pyramid model, tests at all different levels such as unit, integration etc., were given equal priority as end-to-end testing.

There were hundreds of end-to-end tests which were flaky and did not provide any feedback on the code, Haritha said. A comparative study of the available end-to-end tests against all the other tests was carried out. Based on this study, a large number of test cases covered by end-to-end tests were already part of unit and integration tests, thereby providing no additional benefit and hence the end-to-end tests were safely removed.

Further analysis of the remaining end-to-end tests against the test pyramid model was made. These tests were rewritten to the appropriate layers of the testing pyramid, thereby ensuring there was good test coverage. Haritha mentioned that only those end-to-end tests which provided the benefits of end-to-end functional coverage from a user flow perspective covering all the systems, and thereby serving the purpose, were retained.

Haritha shared her learning that before bringing in changes, it was important to ease some burdens so as to provide enough space to accommodate the new changes:

In this case, maintaining the hundreds of end-to-end tests was a burden and only after fixing up the flaky tests and reducing their numbers did the team actually show interest towards the processes such as backlog refinements and defect analysis.

InfoQ interviewed Haritha Hari about shifting quality left.

InfoQ: What issues impacting quality did the organization face?

Haritha Hari: The project was using agile for the development stage, but testing happened in a traditional waterfall model where testers did not play any role until most of the features were dev complete.

Testing was separated out into two stages, the first one called project testing where the end-to-end tests and manual testing mocked the external systems. It was not until the second stage of testing, which happened close to the production release, that the system integration issues popped up. This stage happened after the project testing and close to production release where real systems replaced mocks.

Testing remaining as a waterfall approach within agile caused delayed defect identification, leading to a repeat of analysis and development that affected the timely delivery of products into the market.

InfoQ: What misconceptions existed around agile, and how did you deal with them?

Haritha: Some of the ceremonies such as retros and standups in agile were practised without completely understanding the purpose. The team was more focused on sprint velocity and was hesitant to spend any time on improvements.

Agile also was understood as a way of bringing in more scope at random without much analysis. This approach had to be corrected by bringing in more backlog refinement sessions and defect prioritisation meetings.

Agile approaches were not applied beyond the development area and hence testers most often worked independently from the rest of the team. Bringing the team in a physically closer location and having ice breaker sessions improved the collaboration within the team.

Paying more importance to kickoff sessions and local desk checks further improved this collaboration, hence bringing testing a step further in the project.

InfoQ: What have you learned?

Haritha: It is easier to bring shift left testing into a new project as opposed to into an existing long-running project. When trying to implement changes such as breaking the hundreds of end-to-end tests into unit and integration tests, there was a lot of push back as there existed a lack of awareness on shift left testing approaches and the test pyramid model.

It was then that I realised I had to do some coaching and training before actually implementing a change. The coachings mainly centered around shift left testing and testing strategies.

InfoQ: What are your suggestions for implementing disruptive changes effectively?

Haritha: It is always important to make people aware of the changes and educate them on the need to implement changes. Leading by example in a small part of the project is the best way to implement disruptive change. When such changes bring in a positive impact on the overall work, then it is easier to bring in people who will follow the new way of doing things.

Rate this Article