BT

New Early adopter or innovator? InfoQ has been working on some new features for you. Learn more

Test Well and Prosper: The Great Java Unit-Testing Frameworks Debate

| by Michael Redlich Follow 7 Followers on Jun 20, 2016. Estimated reading time: 2 minutes |

A recent post in Reddit sparked a debate between supporters of JUnit and Spock testing frameworks, based on a blog post by Jakub Dziworski with the central theme, “What’s wrong with JUnit?”. It appears that every GitHub repository includes unit tests built with JUnit, which should come as no surprise since JUnit has been around for over 15 years. Yet, Spock continues to chip away.

JUnit was created by Kent Beck, creator of eXtreme programming, and Erich Gamma,co-author of Design Patterns: Elements of Reusable Object-Oriented Software, and it quickly became a defacto standard in unit testing, with ports and clones in nearly every popular programming language. However, JUnit’s features have been questioned over the years resulting in new unit-testing frameworks, such as TestNG and Spock.

TestNG

TestNG was created by Cédric Beust, co-author of Next Generation Java Testing: TestNG and Advanced Topics, in 2004. According to the TestNG website, “TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use…”. Beust maintains on his website , “I started TestNG out of frustration for some JUnit deficiencies which I have documented on my weblog here and here.”

Spock

In his blog post, Dziworski questions using JUnit combined with a third-party mocking framework. He states, “Java combined with those frameworks makes it rather hard to write and read tests in medium and large sizes projects.” He goes on to say, “If tests are hard to write we usually think of them as something painful and start to neglect them. Avoiding or delaying writing tests leads to the situation where application cannot be trusted anymore. We then become afraid of making any changes because another part of the app might break in some bizarre way.”

At a recent Java Hellenic User Group meeting, Kostis Kapelonis, author of Java Testing with Spock, offered a presentation where he compared JUnit with Spock.

Spock was created by Peter Niederwieser, Principal Engineer at Gradleware, in 2008. While inspired by JUnit, Spock’s features are more than just an extension of JUnit:

  • Tests are written in Groovy which can test code written in Java.

  • A built-in mock framework eliminates the need to import third-party frameworks.

  • The ability to provide custom names for tests.

  • Predefined behavior-driven blocks (given:, when:, then:, expect:, etc.) for establishing tests.

  • Use of data tables eliminates the need to use data structures.

The following code snippet (as shown in the Reddit post) demonstrates some of these features:

1 class Math extends Specification {
2     def "maximum of two numbers"(int a, int b, int c) {
3         expect:
4         Math.max(a, b) == c

5         where:
6         a | b | c
7         1 | 3 | 3   // passes
8         7 | 4 | 4   // fails
9         0 | 0 | 0   // passes
   }
}

This simple test example uses two predefined blocks, expect: (line 3) and where: (line 5). The where: block is used to define a data table that maps to the expectation of the Math.max function defined in line 4. Line 2 demonstrates how a custom name can be defined for a test.

An entire project containing JUnit and Spock code samples can be found on GitHub.

As far back as 2008 InfoQ reported on some speculation of JUnit’s demise. Eight years later JUnit 5 is alive and well and milestone 1 is in development. Test well and prosper!

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Project containing JUnit and Spock code samples ? by Petr Sakar

Are you sure the project is ready for public review ?
Eg. test class name TestHelloSpock, HelloSpockSpec not testing the HelloSpock class, ...

Re: Project containing JUnit and Spock code samples ? by Michael Redlich

If you look at the directory structure, you will see that there are two paths under the test directory, groovy and java.

The java path contains TestHelloSpock which tests HelloSpock under the normal source code path.

The groovy path contains HelloSpockSpec which tests the string. So, essentially, the code to be tested and the test are in one Groovy file.

I will update the project to include a separate Spock test for the HelloSpock class.

I hope that I cleared up any confusion...

Debate? by Johnathon McAlister

Not much of a debate here, so I will start one...

First, I work on a multi-million line project with thousands of classes and a total of about 100 developers in multiple teams. Over the years, things have moved from waterfall with a separate test team doing manual system tests to small agile teams using TDD and automated unit and func testing (we still have system testing - and are looking into ways to automate that).

Along the way, we started with JUnit, did a foray into TestNG, and then went back and standardized on JUnit. We then tried Spock for a while, and occasionally use it - but our gold standard is JUnit. We would use Spock more, but it's use of Groovy interacts with our mixing of Groovy and Java code in odd ways - we usually figure it out, but it makes the tests hard to red and write than one would expect, which negates that benefit over JUnit. TestNG was not compatible enough with our existing tests, and we did not want to go back and "fix" them all. In addition, it's easy to find people who know JUnit - not so much the others.

So - in the end, JUnit gets my vote. Looking forward to JUnit 5, and it's integration into Eclipse.

Re: Debate? by Michael Redlich

Hi Johnathon:

Thanks for sharing your experiences with JUnit, TestNG, and Spock on a very large project with so many developers. I'd be curious to know more about the odd ways your project was affected by mixing Groovy and Java.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

4 Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT