BT

InfoQ Homepage News Test Well and Prosper: The Great Java Unit-Testing Frameworks Debate

Test Well and Prosper: The Great Java Unit-Testing Frameworks Debate

Bookmarks

A recent post in Reddit sparked a debate between supporters of JUnit and Spock testing frameworks, based on a blog post by Jakub Dziworski with the central theme, “What’s wrong with JUnit?”. It appears that every GitHub repository includes unit tests built with JUnit, which should come as no surprise since JUnit has been around for over 15 years. Yet, Spock continues to chip away.

JUnit was created by Kent Beck, creator of eXtreme programming, and Erich Gamma,co-author of Design Patterns: Elements of Reusable Object-Oriented Software, and it quickly became a defacto standard in unit testing, with ports and clones in nearly every popular programming language. However, JUnit’s features have been questioned over the years resulting in new unit-testing frameworks, such as TestNG and Spock.

TestNG

TestNG was created by Cédric Beust, co-author of Next Generation Java Testing: TestNG and Advanced Topics, in 2004. According to the TestNG website, “TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use…”. Beust maintains on his website , “I started TestNG out of frustration for some JUnit deficiencies which I have documented on my weblog here and here.”

Spock

In his blog post, Dziworski questions using JUnit combined with a third-party mocking framework. He states, “Java combined with those frameworks makes it rather hard to write and read tests in medium and large sizes projects.” He goes on to say, “If tests are hard to write we usually think of them as something painful and start to neglect them. Avoiding or delaying writing tests leads to the situation where application cannot be trusted anymore. We then become afraid of making any changes because another part of the app might break in some bizarre way.”

At a recent Java Hellenic User Group meeting, Kostis Kapelonis, author of Java Testing with Spock, offered a presentation where he compared JUnit with Spock.

Spock was created by Peter Niederwieser, Principal Engineer at Gradleware, in 2008. While inspired by JUnit, Spock’s features are more than just an extension of JUnit:

  • Tests are written in Groovy which can test code written in Java.

  • A built-in mock framework eliminates the need to import third-party frameworks.

  • The ability to provide custom names for tests.

  • Predefined behavior-driven blocks (given:, when:, then:, expect:, etc.) for establishing tests.

  • Use of data tables eliminates the need to use data structures.

The following code snippet (as shown in the Reddit post) demonstrates some of these features:

1 class Math extends Specification {
2     def "maximum of two numbers"(int a, int b, int c) {
3         expect:
4         Math.max(a, b) == c

5         where:
6         a | b | c
7         1 | 3 | 3   // passes
8         7 | 4 | 4   // fails
9         0 | 0 | 0   // passes
   }
}

This simple test example uses two predefined blocks, expect: (line 3) and where: (line 5). The where: block is used to define a data table that maps to the expectation of the Math.max function defined in line 4. Line 2 demonstrates how a custom name can be defined for a test.

An entire project containing JUnit and Spock code samples can be found on GitHub.

As far back as 2008 InfoQ reported on some speculation of JUnit’s demise. Eight years later JUnit 5 is alive and well and milestone 1 is in development. Test well and prosper!

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Project containing JUnit and Spock code samples ?

    by Petr Sakar /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Are you sure the project is ready for public review ?
    Eg. test class name TestHelloSpock, HelloSpockSpec not testing the HelloSpock class, ...

  • Re: Project containing JUnit and Spock code samples ?

    by Michael Redlich /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    If you look at the directory structure, you will see that there are two paths under the test directory, groovy and java.

    The java path contains TestHelloSpock which tests HelloSpock under the normal source code path.

    The groovy path contains HelloSpockSpec which tests the string. So, essentially, the code to be tested and the test are in one Groovy file.

    I will update the project to include a separate Spock test for the HelloSpock class.

    I hope that I cleared up any confusion...

  • Your message is awaiting moderation. Thank you for participating in the discussion.

    Not much of a debate here, so I will start one...

    First, I work on a multi-million line project with thousands of classes and a total of about 100 developers in multiple teams. Over the years, things have moved from waterfall with a separate test team doing manual system tests to small agile teams using TDD and automated unit and func testing (we still have system testing - and are looking into ways to automate that).

    Along the way, we started with JUnit, did a foray into TestNG, and then went back and standardized on JUnit. We then tried Spock for a while, and occasionally use it - but our gold standard is JUnit. We would use Spock more, but it's use of Groovy interacts with our mixing of Groovy and Java code in odd ways - we usually figure it out, but it makes the tests hard to red and write than one would expect, which negates that benefit over JUnit. TestNG was not compatible enough with our existing tests, and we did not want to go back and "fix" them all. In addition, it's easy to find people who know JUnit - not so much the others.

    So - in the end, JUnit gets my vote. Looking forward to JUnit 5, and it's integration into Eclipse.

  • Re: Debate?

    by Michael Redlich /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Hi Johnathon:

    Thanks for sharing your experiences with JUnit, TestNG, and Spock on a very large project with so many developers. I'd be curious to know more about the odd ways your project was affected by mixing Groovy and Java.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT

Is your profile up-to-date? Please take a moment to review and update.

Note: If updating/changing your email, a validation request will be sent

Company name:
Company role:
Company size:
Country/Zone:
State/Province/Region:
You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.