The state of testing survey aims to provide insights on a number of aspects of the testing profession. Reviewing things like the adoption of test techniques and practices, test automation, and many of the other challenges that testers are facing today. The survey, made by testers for testers, is organized by Joel Montvelisky from PractiTest together with Lalit Bhamare from Tea-Time with Testers.
2016 will be the third time that this survey is done. The survey questions will follow in the steps of the ones used in the previous surveys as Montvelisky explains:
Joel Montvelisky: We are still in the lookout for trends in the world of testing and so we are trying to stick with questions that are similar in order to help us compare between the data of the 3 surveys we will have by the end of this one.
In the interview with Montvelisky about the state of testing 2013 report he talked about the agile practices, techniques and tools that were mostly used by testers:
Joel Montvelisky: What we do see is that Agile teams have higher percentages of automation than "non-agile" testing teams. We see many different automation tools being used, from Selenium to TestComplete and everything in between.
Another thing we see a lot is that teams are running their test automation as an integral part of the development process and not as an isolated testing operation. We see many automation testers working with the same development frameworks such as Jenkins and Bamboo to integrate their testing directly into the build process.
Regarding techniques, we see a lot of Exploratory Testing together with Scripted Testing. This blend helps teams to cope with the dynamic demands of their project while still maintaining a level of security that comes from having a defined but maybe more limited formal testing suite.
The state of testing 2015 report mentioned the main challenges that test teams were facing at that time. From the InfoQ interview with Joel Montvelisky and Lalit Bhamare:
Joel Montvelisky: I guess the report makes it very clear that the top most challenges of teams and their managers are around growth and time.
Growth is reflected on the challenge they report in hiring good testers, and in principle this is a good news for the industry as challenges in hiring can be translated into a higher demand for good testers (this is for all of you who thought testing, and manual testing, was a dying profession...).
The second main challenge is around timeframes. Again, I guess this is related to the fact that the world is leaning towards more lean and agile practices, and this approach challenges our previous assumptions of long and comprehensive testing cycles, and places us on a reality where we need to work more based on risk analysis, automation, development testing, and overall less time to test.
Lalit Bhamare: Summing that up in one line, I would say the main challenge is "doing more with less". That means doing more (and meaningful) testing in less time, finding more defects in less time, doing effective/more testing in less budget, doing effective testing with less testers, doing effective testing with less available information and so on.
I’m not sure if majority of testers have found ways to deal with it but among those I know, they are solving such problems by following methodologies like Rapid Software Testing and principles of Context Driven Testing i.e. by becoming a "thinking tester".
InfoQ asked Montvelisky about trends that he sees are happening in testing, on which he expects the survey to provide insight:
Joel Montvelisky: I think that some of the things we are looking for (at least unconsciously) is to see how much the industry as a whole is moving towards slimmer practices, how much of the testing tasks are being integrated into the core agile teams, to see if the trend we saw towards Agile is becoming now a push towards DevOps, and also to see how people continue moving towards social networks and virtual conferences as a means of keeping up to date with the trends and knowledge in testing today.
But I believe that most of the conclusions will actually come from the results of the survey themselves. We are still asking a wide range of questions that will help us to get more in-depth into the reality of the testing profession and our community.
If you participating in the state of testing survey then you will receive a complimentary copy of the state of testing 2016 report once it is published.