Survey: Devs Are the Main Roadblock in Adopting TDD/BDD

| by Abel Avram Follow 12 Followers on Jul 29, 2016. Estimated reading time: 2 minutes |

QASymphony, a testing services company, has recently released the State of Test-First Methodologies 2016 Report, a survey of over 200 people/organizations from 15 countries. The purpose of the survey was to evaluate the adoption of test-first technologies -BDD/ATDD/TDD –  and how they are perceived by respondents.

The main takeaways that we have extracted from the report are:

Nearly half of the respondents have not implemented a BDD/ATDD/TDD approach. Regarding those who have adopted such technologies, 37% have done it in the last year, and only a small minority (~13%) have up to 3 years or more practice in this field.

From those who have implemented a test-first solution to software development, almost half have done it to increase the quality of their software, 23% have done it for a better collaboration between team members and increased understanding of product requirements, 12% for faster delivery and 8% to have more automation.

In half of those organizations (52%), developers along with testers are responsible for writing tests, while in other 40% of them only those assigned as testers. In only a small fraction (2.7%) of software makers developers are fully responsible for writing tests.

When it came to what hinders the adoption of a test-first methodology, the largest group (~44%) cited fear of “forcing developers to contribute to tests before writing code.” It seems there is a strong resentment among certain programmers to the idea of doing something that seems to be “useless” rather than doing what they like, writing code that works toward implementing features. The next group of respondents (~36%) fear changing the existing testing procedures or framework with a new automated testing framework.

45% of the respondents managed to switch to a test-first approach in less than 3 months, while 30% needed up to a year. Some (~12%) needed 3 years or more. Regarding the suggestions they would make to others interested in test-first, the survey concludes that the cultural factor is paramount:

Getting the entire team on board as well as all levels of the organization is key to gaining the understanding and collaboration needed to make the test-first shift successful. Additionally, patience is key, both in finding the right sized project to get started as well as setting realistic goals and waiting long enough to see results. Overall, it seemed like technical skills and tool implementation fell low on the list of concerns for most organizations looking in the rearview mirror on a test-first implementation.

Respondents use a large variety of test-first tools, mostly open source. JUnit has preeminence among unit testing tools, and Cucumber among BDD ones.

For those interested in reading more findings, such as what other process have people used before, what they think is the next big thing in testing, what they fear about TDD/BDD, etc., we recommend reading the entire report.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Another annoying cult forced on developers by Hermann Schmidt

I have just finished reading "Why Do So Many Programmers Hate Agile?" on DZone and the well-deserved shit-storm on real-world "Agile" in the comments and now I read this.

What a pain in the butt! Developers are blocking the promise of salvation of some self-proclaimed "thought leaders"? The stupid developers again. We really need to tell those infantile beings how to do their work every single minute! You cannot trust these people. They need to be managed strictly and in a tight loop!

I am so sick and tired of this. I program for over 30 years now and guess what, I do not TDD because that's not how my brain works! I write tests where they are necessary and when I need to verify myself, but otherwise I let myself go with the flow because that is how my creativity unfolds.

Instead of forcing developers into a new religion every 5 years, let them do their job as they feel it is right for them. Continuously qualify them to enlarge their toolset. Let them decide how they code.

Quality arises from qualification, not from rituals.

Re: Another annoying cult forced on developers by Abel Avram

What a good laugh I had reading your comment. :) When I was writing this news, I noticed their question "What do you fear when it comes to implementing a test-first approach?", and 44% said "developers". I said to myself "That is a bit far-stretched. Fear is too strong of a word." Your comment proves they were right to be fearful. :)

I hope you noticed that this article is not advocating TDD. I was simply reporting on the findings of a survey. That being said, I would comment a bit on your comment. Why are you thinking that a manager introduces TDD or Agile in a company because he considers the developers as "stupid" or "not trustworthy"? And why shouldn't TDD/Agile be a part of the qualifications you mentioned?

TDD/Agile are not about the C#/Java/etc. code that people write, but about how they write it, and how they do it as a team. Writing a program can be done in many ways and languages: from writing ASM alone on a piece of paper to distributed teams using XP in a cloud IDE along with CI/CD and all sorts of other methodologies. They are just different ways to reach the same goal. Are some ways better than others? Some believe not all programming methods are equal. And I believe too. I think the days of the lonely and highly qualified programmer that does the job by himself are pretty much over. Some companies are moving so fast these days that are leaving others behind in dust. Why? Because their teams are very well organized, with people understanding the importance of working together and having a process that helps them do that. I'm not saying that Agile is the best answer to those needs, but the world is looking for ways to improve how things are done. We are in a process of discovering the answer. And perhaps it is not a one-size-fits-all answer. Perhaps there are multiple answers. One thing is sure: working as a team is a process much more complex than working alone, a process that requires participants to change. A lot. It's what some call "cultural change". Those who manage to adapt to change, and find a proper balance between the individual and the team, will be the ones who succeed. The others will just write code that gets thrown to the garbage can, as it happens with most of the software written today.

It is not about "stupid" vs. "smart" developers, it is about teams and we approach development as a team.

Re: Another annoying cult forced on developers by Hermann Schmidt


I wasn't critisising you article, you just did a report. I was pre-loaded by the article I've read before and my rant picked on a narrow aspect of SW development.

TDD needs to be experimented with. I did that and I decided to not to use it judiciously. I can deliver high quality code with a different coding strategy. I seek my own faults, that's one pillar of agility.

You see, patterns repeat themselves. Even when devs do not choose TDD, management will come in totally unqualified, fed with buzzwords, and say "I have heard it improves code quality. <Put your favorite company in here> does it, and I want you to do it, too".

I agree with your assessment that the days of the lonely programming heroes are over. I would even go further and say that such characters are bad for an organisation. However, tests do not foster team spirit. I just cannot follow this conclusion. Tests are just code and they can become a liability like all other code. TDD does not miraculously change that fact.

So, what's my take on improving teamwork? Don't tell teams how to do things. Give them the opportunity to learn and decide themselves on techniques. Communicate to them the constraints they need to satisfy to make a project an economic success. Teach them to be open, to look around themselves, not to hide mistakes until it is too late, don't sanction failures if they are identified quickly and honestly.

Call that agile or whatever. It is basically common sense and the only sensible approach to manage knowledge workers.

Teams with stupid (unqualified) people fail. Teams deprived of responsibility fail. The combination of competent individuals who take responsibility makes a company great.

Re: Another annoying cult forced on developers by Wen Hu

I'm very much in agreement with what has been commented.
Testing services companies in general over-emphasize or exagerate the distinctness and generality of testing. Testing is a means to obtaining higher quality. It is important of course but there is so much more that is of importance (requirements, design and such). Subject matter connects all these things. In my experience most testing services consultants do not keep up with requirements specifiers and developers in this regard.

Re: Another annoying cult forced on developers by Abel Avram

Yes, TDD is not directly related to team collaboration. I extended the discussion to Agile because you mentioned it. Although, it is great when joining a new team and they have a safety net of comprehensive tests that gives you some assurance that you are informed when you break things. I'm sure everybody likes the safety of tests, but few have the patience to write them, myself included.

Commenting on the management actions that you mentioned, I think we should discriminate between a methodology and how it is applied by a certain manager. A certain methodology is not automatically bad because some manager applies it badly. I think both the team and the manager should be more flexible: the first in accepting change, the second in improving how to introduce change.

It would be great to have a super qualified team of developers that know for themselves how to go about things. But that is rare, and may work in small teams, such as those associated to microservices. But in most cases it helps to introduce change from outside via a consultant or manager. Does it really matter that knowledge comes from within or outside the team? All it matters is the knowledge, not how it comes. If we are open to that we would embrace change no matter where it originates from. Now, is it possible that we get the wrong advice? Yes, it always is, be it from within or outside. A self-organized team may make mistakes as bad as ones enforced by a manager. When advice and subsequent decisions come from inside the team, it is more easily embraced, but there is no guarantee it is the right decision. Only time will tell. But being vulnerable to mistakes does not mean we should stop learning and increasing our knowledge. It is all part of learning.

Re: Another annoying cult forced on developers by Hermann Schmidt

With "opportunity to learn" I did not rule out consultancy from outside. Of course this is a valid strategy for learning.

From my experience though, at least some team members have heard of things long before managers did. So, let them go and try it and offer help if necessary. They do know what's out there! Let some team members lead the initiative and take responsibility. No leaders available? Then the team is incomplete and needs help.

When the team's decision satisfies the constraints given by management, the decision is right as long as those constraints are valid. That's how management should measure it. Not by ticking checkboxes like "have TDD, done Scrum, according to plan".

Re: Another annoying cult forced on developers by Kevin Dunne

Appreciate all of the lively discussion here on our survey. I do see both sides of the argument - on the one hand, TDD can seem like an authoritarian approach and many times developers feel it does not provide them direct value (ex. write a test to confirm that the login button has moved to the right). On the other hand, TDD does seem to create the most value when implemented consistently across the board. When the team cannot have faith that all developers are following it, they cannot be sure what it means when the automated test suite passes. Is the code really working, or could it be that many of the tests are incomplete/no robust, or even worse, missing altogether?

I think that any process that takes time away from direct development and creative activity will be questioned and find some resistance in the developer community. Many of the benefits of TDD are not directly provided to the developer, but rather the team and organization as it scales. So it is not uncommon at all to find that individual developers are not interested in the additional effort when there is no immediate benefit to them.

As Abel mentioned, we do not have a strong opinion on whether or not organizations should be using a TDD approach. We just hope to enlighten users who are considering or in the midst of adopting TDD about what others in the industry are doing. What we do know is that TDD is still very much misunderstood and the amount of reliable information available about it is still low.

Re: Another annoying cult forced on developers by Raimo Jormakka

I guess different people also have different ideas of what TDD actually means. You seem to be thinking that a team is doing TDD just by maintaining an automated test suite but, if you are strict about it, TDD is actually about maintaining that test suite in a specific way. I don't know any developers who don't see value in writing tests but I know many who don't see the value of TDD.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

9 Discuss