The State of Testing in 2016

by Ben Linders on Jan 08, 2016 |

The state of testing survey aims to provide insights on a number of aspects of the testing profession. Reviewing things like the adoption of test techniques and practices, test automation, and many of the other challenges that testers are facing today. The survey, made by testers for testers, is organized by Joel Montvelisky from PractiTest together with Lalit Bhamare from Tea-Time with Testers.

2016 will be the third time that this survey is done. The survey questions will follow in the steps of the ones used in the previous surveys as Montvelisky explains:

Joel Montvelisky: We are still in the lookout for trends in the world of testing and so we are trying to stick with questions that are similar in order to help us compare between the data of the 3 surveys we will have by the end of this one.

In the interview with Montvelisky about the state of testing 2013 report he talked about the agile practices, techniques and tools that were mostly used by testers:

Joel Montvelisky: What we do see is that Agile teams have higher percentages of automation than "non-agile" testing teams. We see many different automation tools being used, from Selenium to TestComplete and everything in between.

Another thing we see a lot is that teams are running their test automation as an integral part of the development process and not as an isolated testing operation. We see many automation testers working with the same development frameworks such as Jenkins and Bamboo to integrate their testing directly into the build process.

Regarding techniques, we see a lot of Exploratory Testing together with Scripted Testing. This blend helps teams to cope with the dynamic demands of their project while still maintaining a level of security that comes from having a defined but maybe more limited formal testing suite.

The state of testing 2015 report mentioned the main challenges that test teams were facing at that time. From the InfoQ interview with Joel Montvelisky and Lalit Bhamare:

Joel Montvelisky: I guess the report makes it very clear that the top most challenges of teams and their managers are around growth and time.

Growth is reflected on the challenge they report in hiring good testers, and in principle this is a good news for the industry as challenges in hiring can be translated into a higher demand for good testers (this is for all of you who thought testing, and manual testing, was a dying profession...).

The second main challenge is around timeframes. Again, I guess this is related to the fact that the world is leaning towards more lean and agile practices, and this approach challenges our previous assumptions of long and comprehensive testing cycles, and places us on a reality where we need to work more based on risk analysis, automation, development testing, and overall less time to test.

Lalit Bhamare: Summing that up in one line, I would say the main challenge is "doing more with less". That means doing more (and meaningful) testing in less time, finding more defects in less time, doing effective/more testing in less budget, doing effective testing with less testers, doing effective testing with less available information and so on.

I’m not sure if majority of testers have found ways to deal with it but among those I know, they are solving such problems by following methodologies like Rapid Software Testing and principles of Context Driven Testing i.e. by becoming a "thinking tester".

InfoQ asked Montvelisky about trends that he sees are happening in testing, on which he expects the survey to provide insight:

Joel Montvelisky: I think that some of the things we are looking for (at least unconsciously) is to see how much the industry as a whole is moving towards slimmer practices, how much of the testing tasks are being integrated into the core agile teams, to see if the trend we saw towards Agile is becoming now a push towards DevOps, and also to see how people continue moving towards social networks and virtual conferences as a means of keeping up to date with the trends and knowledge in testing today.

But I believe that most of the conclusions will actually come from the results of the survey themselves. We are still asking a wide range of questions that will help us to get more in-depth into the reality of the testing profession and our community.

If you participating in the state of testing survey then you will receive a complimentary copy of the state of testing 2016 report once it is published.

Rate this Article


Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Testing in the new age by Susan Obijiski

You make some good points here. As testing evolves, it is important to remember that the fundamentals of the process must not be overlooked. With so many new bells and whistles on apps and software products and the need for a good Ux and great mobile performance, the wise IT consultant and/or IT team must plan for thorough testing that covers all dynamics and use and avoid the mistake of trying to cut cost or time by cutting back on due diligence. At Elegant MicroWeb we also find that involving clients in testing can be critical. They often have valuable input and can provide focused user testing with an eye toward what their customers and their users might expect and, in so doing, they often ask questions and uncover expectations an IT team might not otherwise anticipate.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

1 Discuss
General Feedback
Marketing and all content copyright © 2006-2016 C4Media Inc. hosted at Contegix, the best ISP we've ever worked with.
Privacy policy

We notice you're using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.