InfoQ

Presentation

Recorded at:
Recorded at

Testing is Overrated

Presented by Luke Francl on Jan 02, 2009

Community
Agile,
Ruby
Topics
Delivering Quality ,
Software Testing ,
Defects
Tags
Testing ,
Code Reviews ,
RubyFringe
Summary
Developer-driven testing is probably the most influential software development technique of the last 10-15 years. There's no question that it has improved the practice of building software. And in a dynamic language like Ruby, it's hard to get by without it. But is it really the best way to find defects? Or is the emphasis on testing and test coverage barking up the wrong tree?

Bio
Luke Francl is a developer at Slantwise Design (http://slantwisedesign.com) and sometimes tech writer. Luke is a frequent presenter at the Ruby Users of Minnesota, and has also presented at CodeCon, MinneBar, Ostrava on Rails, and acts_as_conference. He blogs at Rail Spikes (http://railspikes.com) and Just Looking (http://justlooking.recursion.org).

About the conference
RubyFringe is an avant-garde conference for developers that are excited about emerging Ruby projects and technologies. They're mounting a unique and eccentric gathering of the people and projects that are driving things forward in our community.

Related Sponsor

VersionOne is recognized by Agile practitioners as the leader in Agile project management tools. Companies such as Adobe, BBC, CNN, Dow, HP, IBM, Sony and 3M have turned to VersionOne to help deliver greater value to their customers.

Missed the point by Channing Walton Posted Jan 17, 2009 3:13 PM
Re: Missed the point by Clinton Begin Posted Jan 18, 2009 8:28 PM
Re: Missed the point by Franz Allan See Posted Jan 19, 2009 7:58 AM
Re: Missed the point by Markus Kohler Posted Jan 19, 2009 4:11 PM
Interesting by Prajwal Tuladhar Posted Jan 18, 2009 12:53 AM
Progress? by Christoph Henrici Posted Jan 19, 2009 2:07 AM
testing is just a substitute by Eirik Maus Posted Jan 19, 2009 12:27 PM
Value of testing by Tony Ambrozie Posted Jan 19, 2009 10:59 PM
  1. Back to top

    Missed the point

    Jan 17, 2009 3:13 PM by Channing Walton

    TDD is not just about testing, it’s a design technique.

  2. Back to top

    Interesting

    Jan 18, 2009 12:53 AM by Prajwal Tuladhar

    Interesting presentation and good sense of humor while presenting

  3. Back to top

    Re: Missed the point

    Jan 18, 2009 8:28 PM by Clinton Begin

    Agreed.

  4. Back to top

    Progress?

    Jan 19, 2009 2:07 AM by Christoph Henrici

    It is really astonishing where the scripting languages are getting too and selling it progress....

  5. Back to top

    Re: Missed the point

    Jan 19, 2009 7:58 AM by Franz Allan See

    (Unit) Testing is not about getting all bugs at once. It's about getting (and fixing) bugs fast (even if you don't do TDD).

    When you write a code with test, other developers can go in and change it and see if they broke anything functionality that you did.

    And if during the course of usage, a bug arises, you can debug your code easier if you have tests all around. Because those states who are not tested are most likely be the suspect (thereby decreasing the states that you have to go through).

    Having unit tests will not guarantee a bug-free system. But it should speed things up in fixing them.

  6. Back to top

    testing is just a substitute

    Jan 19, 2009 12:27 PM by Eirik Maus

    ... for real-world feedback. It is a substitute that is repeatable and instantaneous and with no damage done.

    Exposure to the real world is actually better, since it better (very effective, actually) to find errors in the specifications. However, making errors in the real world, when you are developing money-moving software with more than 10 digits for the amount, is not so practical.

  7. Back to top

    Re: Missed the point

    Jan 19, 2009 4:11 PM by Markus Kohler

    Agreed.
    Without good unit testing, refactorings are suicide.

  8. Back to top

    Value of testing

    Jan 19, 2009 10:59 PM by Tony Ambrozie

    I will take this (humorous but well argued) presentation as really a re-affirmation of the valuable but within-limits benefits of unit testing. As with any good thing, we have to be careful to clearly set expectations of its limitations: I agree that metrics get sometimes out of hand and become meaningless, I also agree that you can't test what wasn't implemented and should have and that unit code testing does not replace end-user testing.

    With that said, code testing does improve quality of implemented code and features (in my own experience), it provides a firm base to validate the aging code (as during refactoring) and new features against and, not lastly, provides a form of code documentation (most importantly the intention behind the code -- not just mere comments in code -- which will be valuable years later, as some code runs and it is maintained by many people for long periods of time.) I would not discard here the sense of confidence good tests passed inspire in developers, especially important during difficult project periods. I would also think that scripting/dynamic languages need more testing because of the absent/reduced compiler-supported validation. The problem with insufficient negative case testing -- testing for what could go wrong -- comes from the fact that, in reality, too little attention is given to that -- starting with requirements, use cases, etc -- at a project/product technical and management level, and that is something we can do something about.

    I have not seen code reviews ensure quality of code in a sustainable manner, either as a code reviewer or reviewee -- unless if that is part of pair-programming -- but that could be subjective.

    To conclude, I believe we should be doing (more) testing but understand and act upon its limitations and benefits vs. costs for each particular situation.

Educational Content

Brian Marick on 4 Challenges and 5 Guiding Values of Agile Software Development

Brian Marick takes us through a quick tour of the most important values and challenges to adopting Agile successfully (they aren't the typical challenges and values we hear in the community).

Are You a Software Architect?

The line between development and architecture is tricky. Does it exist at all? Is an ivory tower actually needed? There's a balance in the middle, but how do you move from developer to architect?

Agile – A Way of Life and Pragmatic Use of Authority

The word 'authority' sometimes produces an allergic response in hard-line agilists. Freedom and authority – both are bad if misused and both are good if used in right spirit for a noble cause.

Getting Started with Grails, Second Edition

"Getting Started with Grails" brings you up to speed on this modern web framework. Companies as varied as LinkedIn, Wired, and Taco Bell are all using Grails. Are you ready to get started as well?

Using ITIL V3 as a Foundation for SOA Governance

Those familiar with only ITIL V2 often scoff at the thought that ITIL could serve as a governance framework for SOA. With ITIL V3, the focus of the framework shifted towards service-orientation.

Adrian Colyer on AspectJ, tc Server and dm Server

SpringSource CTO Adrian Colyer discusses AspectJ, SpringSource's dm Server and tc Server products, OSGi and Scrum.

Adam Wiggins on Heroku

Heroku's Adam Wiggins talks about Rails, Background Jobs, Add-Ons, Ruby, and how Heroku manages to work around Ruby's inefficiencies using Erlang and other languages.

SOA as an Architectural Pattern: Best Practices in Software Architecture

For Grady Booch the foundation of a good architecture is patterns, SOA being just one of many patterns. In this Second Life presentation, Booch attempts to bring more clarity on what architecture is.