BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Writing Good Unit Tests

Writing Good Unit Tests

Bookmarks

Try to keep units small, use appropriate tools, and pair-up programmers and tester; these are suggestions from Adrian Bolboacă for writing good unit tests. Unit testing is a mixture of programming and testing; programmers can work together with testers to learn from each other and broaden their knowledge horizons.

Adrian Bolboacă, Organizational and Technical Coach and Trainer at Mozaic Works, will talk about different types of automated tests at the European Testing Conference 2017. InfoQ will cover this conference with Q&As, summaries and articles:

[The European Testing Conference] is about getting experts and practitioners together to talk, learn and practice the art of testing. We’re looking into advanced new methods into making our testing more effective, as well as enrich our understanding of fundamental methods to grow a stronger community.

In his blog post Automated Test Purposes, Bolboacă defines what a unit test should do:

A unit test is focused on a method or a class. It should be very small, a few lines of code at the most. There are many ways one can make mistakes when writing unit tests, so it is not a trivial thing. Because they are small they should run in memory and a unit test should run in milliseconds. Any test that touches external dependencies (database, webservice, file system, any I/O) is not a unit test, it is something else (integration test, integrated test, acceptance test, end-to-end test, etc).

InfoQ interviewed Bolboacă about writing good unit tests and using automation in unit testing.

InfoQ: Does it matter who writes unit tests- developers or testers?

Adrian Bolboacă: Unit tests are more technical, they often focus on coding details or even on program language specific concepts. Unit tests look different in a static programming language like Java or in a dynamic programming language like Ruby. That is why the main responsible for unit tests should be a programmer.

On the other hand, testers know a lot better how to make a test plan, to identify interesting values for tests by specific analysis like Equivalence Partitioning or Boundary Value Analysis. So a programmer needs to "steal" this kind of knowledge from testers, or they could pair-up and work together on the tests to write, but then the programmer would implement them.

In my experience pairing is the best option, as both testers and programmers learn more from each other and they broaden their knowledge horizons.

InfoQ: When should you do unit testing?

Bolboacă: The team can write unit tests after the production code was written. We call that test after. But that is often difficult, because the production code needs to be written having testability in mind. If we chose this approach, we need to pass the production code through a process of code review and make sure it is testable. Only after that we can continue with creating the tests in the programmer-tester pair.

There is also the test-first approach, which I recommend to programmers who "stole" a lot of testing knowledge from their peers. Using this approach we start with an analysis of the problem and then write one unit test, the simplest implementation code for it, one unit test, implement it, and so on. When the team reaches this level, I would say that the programmer who is writing the unit tests is half-tester, because doing test-first requires a lot of testing knowledge. In this case the testers would focus on reviewing the unit tests at the end, and writing acceptance tests.

Gil Zilberfeld talks about the benefits that a test first approach can bring in the interview Test First Approaches:

Test First defines what needs to work. It defines what code we need to write to solve specific problems, because we have a definition in a form of a test. It’s fairly simple to know if we have working functionality just by running the tests.

Working this way results in much more coverage, because testing becomes a first-class development activity, rather than getting pushed to the end.

In addition, when writing these tests, and specifying the scenarios, we explore the problem space more, because many questions will come up. In Test-After these discussions sometimes never happen, and the developers code what they think, rather than what the solution needs.

Eli Lopian explained in the article The Day the QA Department Died how unit testing can be a possible QA killer:

Unit testing is a method of testing your specific piece of code to make sure that it is working properly and will fit correctly into the software puzzle. It has been shown that with unit testing you can properly check over 90 percent of your code, and, unlike QA’s manual testing tools, unit tests that are built correctly and tested automatically can evolve with your codebase, testing the code in real-time.

InfoQ: How can you use automation in unit testing?

Bolboacă: Unit testing is a process of analysis that leads to a test plan, and then that test plan can be automated.

In order to automate these tests there are some useful things to have in mind:

  1. Use domain language for tests names
    Often we write the test and that’s all. But code is more often read than written. So be nice to your colleagues and your future self and make the test name clear.
    Use names from the domain, not technical names. A test name like "ExceptionOnOverflow" or "TestThree" or "CustomerTest" is not clear. Instead we should write clear names like "WhenTooManyPlayersAreAddedAnErrorIsReturned", "ACustomerNeedsAllFieldsValidated", "ValidCustomerCanBeUsedByOrder", "InvalidCustomerIsRejectedByOrder". In this way all the people who know the business domain will understand what you are testing there. Even your customer.
  2. Have short tests
    A unit test is short, clear, with a single purpose. The best test has 3-4 lines of code, and that makes it very clear for anybody who reads it.
    We need to have many small unit tests like this, and they will work together like a Product Immune System, where each test is like an immune cell . If a bug appears, one small test should tell you exactly where the problem is. In this way you will have a fast feedback and an easy development and testing cycle.
  3. One assert per test
    If you have more than one assert per test you are testing more than one thing. In this case test names become weird and unclear, tests become too long and the feedback in again unclear; you never know which of the asserts passed or failed.
    Let’s say you have three asserts one after the other. If the first assert fails, the last two will never be checked. You change some production code, and you are not protected by the last two asserts when changing the code. In this case you just have the illusion of a safety net and regression testing.
  4. No chained tests
    I often see the habit of chaining the tests. The reason usually is that the arrange part is very difficult to set up. But this is not a solution. The tests that depend on the other ones in the chain will fail to run most often because of test on the top of the chain. In this way you might change code to make the tests pass, but you might introduce defects that are not validated by the incapacitated tests.
    Always tests should be independent of each other, like an immune cell in the Immune System. They all depend on the Product, and not between each other.
  5. Use the appropriate tools
    There are many tools there: testing frameworks, mocking frameworks, test runners, performance testing tools, security testing tools, etc. Make sure you use the appropriate tool for the job. Don’t just use the tool you know. Often the xUnit frameworks are very good for doing most types of automation, but use specialized frameworks for performance and security. If you want good tests that can become executable specifications, you can use xUnit as well, but maybe you want to use some BDD frameworks that make your life easier.
    Remember that choosing a testing framework is a decision that has to be made on medium and long term. Imagine the costs of learning, and also the costs of usage and maintenance.

Because in the current market there is the need to deliver features faster and faster, we need to automate in a smart way the validation process of the product. This is why automation is essential nowadays.

InfoQ: Which tips do you have for writing good unit tests?

Bolboacă: Unit testing comes from industrial production, and most people forget that. In industry we need to test each small part, and if it is quality compliant we can use it in the next step to assembly the parts. In software we don’t have such a clear definition of what a unit is; there are several opinions- method, class, module. The first difficulty is making sure all the team members decide together what a unit is, and write the test accordingly. For me a unit is very small, the size of a screw in an engine or a nail for furniture. So my tip is to define what a unit is, and focus during the tests review phase on enforcing that decision.

Unit testing is a mixture of programming and testing. Programmers need to know many testing concepts in order to write simple, fast, maintainable tests. My recommendation for programmers is to learn from their colleagues testers about some essential concepts: Equivalence Partitioning, Boundary Value Analysis, Test Coverage, Positive Testing, Negative Testing.

Another very often forgotten part about unit testing is analysis. We don’t start writing tests immediately without thinking. We need to analyze the problem, split it if possible and then think about what unit tests we need to write. A good piece of advice here is to always start with the outputs of the system, and then identify the possible inputs that generate that output. Thank you Chris Matts, for teaching me that.

When writing a unit test, we should use words from the domain of the problem. The test should represent the reason why a feature exists in your product. A person that knows the business domain, but does not know programming, should be able to read your test. Pairing with analysts is very useful in that direction.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • underscores rather than camel casing

    by John Broderick,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    It's a really good idea to have long descriptive names for test as mentioned above. But if you do this, using underscores between the words makes it much easier to read.

    e.g. WhenTooManyPlayersAreAddedAnErrorIsReturned
    vs
    when_too_many_players_are_added_an_error_is_returned

  • Maintainable tests

    by Paul Warren,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Maintainable tests are key and tools like Ginkgo4j (com.github.paulcwarren:ginkgo4j) with it's RSpec-inspired testing framework aid developers in creating human-readable, highly contextual tests that they themselves but more importantly others can maintain with ease.

  • Re: underscores rather than camel casing

    by Adi Bolboaca,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I saw teams writing successfully tests with both camel case and underscore. People tend to be annoyed by the need to always press shift when writing underscores, but for that there are addons that write underscore when you type space like live templates on PHPStorm or ReSharper.

    The test names should be the same for the whole product, and the team should decide. For me it is a question of style, I don't mind either camel case or underscores, as long as they are consistent along the project.

  • Re: underscores rather than camel casing

    by Ben Linders,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    To add to this discussion, names should be descriptive, but still as short as possible making it easier to read them. The aim is not to have long names.

    E.g.
    CheckTooManyPlayers or TooManyPlayers
    i.s.o.
    WhenTooManyPlayersAreAddedAnErrorIsReturned

    I have the same experience as Adi on keeping things consistent. Design/coding rules can help here. Use them in your reviews, when pairing, or coaching new team members.

  • What is a unit?

    by Stephen Childs,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I prefer Martin's answer as to what a unit is martinfowler.com/bliki/UnitTest.html

  • Re: What is a unit?

    by Adi Bolboaca,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Martin's definition of a unit test is very good. I see no contradiction here. On the other hand this article was about writing good unit tests and Martin's article is about what is a unit.

  • Re: underscores rather than camel casing

    by John Broderick,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    If making the tests easier to read and write is your aim, then I think we're on the same page.

    My reasoning behind this approach is this, if I want to tell someone about a system, a complete english sentence is going to be the clearest way to communicate it. Due to restrictions in most lanaguages, we cannot have spaces in names, so an underscore is a good alternative. This is the form that we will use when reasoning about the system in our heads and talking about it with others, so why not write them like this?

    Trying to keep names a reasonable length is good advice when writing production code. The benefit I see there is not having to write a really long name whenever I call a method and reducing the clutter that this causes on the screen. But writing tests is a very different scenario. A test isn't an object that we are going to create and issue commands to, it's much more like a specification.

    That's why being able to read 'adding_too_many_players_results_in_an_error' is going to communicate much better than 'TooManyPlayers' or AddingTooManyPlayersResultsInAnError.

    If you want to test this, try taking this whole paragraph and re-writing it in camel case and see how easy it is to read. Then try it with underscores. It's still not as ideal as spaces but it's much easier than camel case.

    Of course people can successfully write test names that are both short and in camel case, but if we can make things clearer and simpler, then why not?

  • Re: underscores rather than camel casing

    by Ben Linders,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Yes, test code should be easier to read and write, basically easy to maintain. That is why using good names that explain what is being tested matters. Glad to see we agree on that.

    What I have seen working with teams is that test code is often treated different that code that is part of the product, production code. This is something that worries me.

    I have seen code being duplicated when tests are created. Large chucks of code to setup conditions for testing a function and lots of code to validate the results which results into large unreadable test modules. Inconsistency in names, formatting, etc, in test code which doesn't seem to worry anyone.

    Although there are differences between test code and production code, there are much more similarities so I prefer to treat them the same. Which includes having the same approach for naming functions. This is where I have difficulty with your suggestions to allow test function names to become long, where you don't want that for production code.

    In the Q&A Adi mentioned that developers and tester can support each other and learn form each other. How to develop good quality code, code that is readable and maintainable, is something that many developers know how to do. Testers, particularly those who are new to testing, might not have these skills. Developers can help them. I agree with Adi that pairing is a great way to learn good programming skills.

  • Re: underscores rather than camel casing

    by Adi Bolboaca,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    As I said, underscores or not, that is a question of style. I know more or less the same number of people that hate one or another, so I am fine with each one.

    Regarding naming, this is a difficult topic. Probably one of the most difficult topic in programming is naming.

    I start giving tests long, clear names that explain clearly, as a specification, what the test does. After that I try to make it shorter and shorter, but without losing the meaning. There are two main techniques that I like:
    - The Continuuum of Naming (naming never ends, it just becomes better as we understand better the domain) that JB Rainsberger wrote about here blog.thecodewhisperer.com/permalink/the-continu...
    - Intentional Programming (use names that express your intention as a programmer) wiki.c2.com/?IntentionalProgramming
    Both of these techniques are very difficult, and we need to enhance our abilities as writers; use correct words, nouns, verbs, etc in each context and focus on clarity.

    At the end I care about test names being clear, and length is part of clarity. The longer, the more difficult to understand. Where is that limit? I don't know exactly, but I observed it is good to have maximum 7-9 words in a test name, the reason being that we keep in our heads around 7 concepts at once en.wikipedia.org/wiki/The_Magical_Number_Seven,...

  • Re: underscores rather than camel casing

    by Ben Linders,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    wrt camel case and underscores, I have no preference for either one of them. Using the same style matter and being consistent matters more to me.

  • Re: underscores rather than camel casing

    by John Broderick,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Not seeings tests as important as the code they are targeting is a concern and any sloppy coding is this area is going to cost the business just like sloppy production code. So yeah, I would treat both of equal importance.

    My point is that designing tests is different to designing a production system so I don't think the same standards and methodologies will fit. That's where I'm coming from with the point on naming, one format suits method names and another format suits test names. It's down to the fact that even though we implement these things using the same language feature, they are conceptually different; one is a command we can issue to an object, the other is a specification of how a system should work.

    So yes we need to maintain the same quality, but we need to come up with a way that suits the context.

  • Re: underscores rather than camel casing

    by Ben Linders,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Makes sense.

    Do you have some references on how tests can be used as a specification?

    How would you use a specification based approach specifically for unit testing, do you have some examples?

  • Re: underscores rather than camel casing

    by John Broderick,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Dan Norths original post on BDD has a good explanation on the usefuleness of sentences (dannorth.net/introducing-bdd/).

    It's a great post but most people I've spoke about BDD feel they have to be using a framework to realise the benefit. But to me, that's all there is to testing. A clear statement on something the system does along with a piece of code to prove it. Coming up with the clear statement is difficult; I wrote a piece about this a while back (simplestdesign.blogspot.ie/2014/01/what-drives-...)

    As an example, is something like this what you had in mind?

    [Test]
    public void a_transfer_credits_the_target_account_and_debits_the_source_account()
    {
    var source_account = An.account().with_credits_of(200);
    var target_account = An.account().with_credits_of(0);

    source_account.TransferTo(target_account, 200);

    Assert.That(target_account.should_have(200));
    Assert.That(source_account.should_have(0));
    }

  • Re: underscores rather than camel casing

    by Ben Linders,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Thanks for sharing the links and example John! Makes things clear.

    For those interested, there's an interview with Dan North on InfoQ where he dives into BDD: www.infoq.com/interviews/dan-north-bdd

  • An article very successful

    by Florin Anghel,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    This article is a very good one.
    Romania has good programmers.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT