JUnit 5 Alpha Simplifies Unit Testing

The JUnit Lambda Team has recently announced the alpha release of JUnit 5, a new major version for the popular unit testing framework. After a successful crowdfunding campaign to allow for full-time developers to work on the project, changes in JUnit 5 are mostly focused at removing the common obstacles that JUnit 4 presented to developers, while also modifying the framework to allow for easier changes in the future. Integration with build tools and IDEs still needs some further work.

Developers will appreciate the way JUnit 5 has adapted to the way they write unit tests on a daily basis. For instance, although the JUnit FAQ traditionally advised against having multiple assertions in a single unit test, anecdotical evidence shows that developers tended to do this anyway. The risk of including multiple assertions per test is that the test execution would stop as soon as one of them fails, without executing the rest of the assertions. Given this, JUnit 5 now supports grouped assertions, where all the assertions are executed even if one or more of them fail:

@Test
void groupedAssertions() {
    assertAll("address",
            () -> assertEquals("John", address.getFirstName()),
            () -> assertEquals("User", address.getLastName())
    );
}

Code sample from the JUnit 5 User Guide.

Another common cause of discontent was the support for testing exceptions, since none of the several ways available in JUnit 4 can fully cater for developers' needs. On one side, one could use the "expected" option in the @Test annotation to indicate the type of the exception that is expected to be thrown; however, this isn't flexible enough to make more complex assertions, for instance to check the message in the thrown exception. In cases like this, developers can make use of @Rule and ExpectedException, which do support the use of matchers to check the exception message; however, even ExpectedException falls short to test further aspects like custom fields or methods in the exception object, or to execute additional assertions after the method that throws the exception.

To address this, JUnit 5 provides a new assertion, expectThrows, that not only will check that a particular method throws an exception of the expected type, but that will also return the exception object as a result to allow for further testing.

@Test
void exceptionTesting() {
    Throwable exception = expectThrows(IllegalArgumentException.class,
            () -> { throw new IllegalArgumentException("a message"); }
    );

    assertEquals("a message", exception.getMessage());
}

Code sample from the JUnit 5 User Guide.

Extensibility has also been looked at. The main way to extend the behaviour of JUnit in version 4 was through the use of @RunWith. However, @RunWith can only be applied once to any given test class, which means developers couldn't make use at the same time of, for instance, Parameterized to apply different data to the same tests, and MockitoJUnitRunner to automatically set up mocks. In JUnit 5 @RunWith has been substituted by @ExtendWith, a new extension mechanism that does support multiple instances being utilised.

In order to allow developers to migrate to the new version progressively, all classes in JUnit 5 have been moved to new packages, and many of the annotations have been renamed. Moreover, the JUnit Lambda Team have created a JUnit 5 Runner for JUnit 4 that allows using all the features in JUnit 5 while still using JUnit 4 as the main testing framework. This makes possible for JUnit 4 and JUnit5 to coexist in the same project, easing transition periods.

Finally, care has been take to facilitate future changes, even backwards-incompatible ones, through a combination of semantic versioning and annotated APIs. As of JUnit 5, every public class and interface in the testing framework will be marked with one of the following values:

• Internal: not meant to be used out of the framework, may be removed at any moment without prior notice.
• Deprecated: should not be used, might disappear in the next minor release.
• Experimental: intended to gather feedback on new concepts; it may be promoted, or it may be removed without prior notice.
• Maintained: no backwards-incompatible changes from the present version to at least the next minor release; if planned for removal, it would change to Deprecated first.
• Stable: no backwards-incompatible changes during the present major version.

Ten years after the last major release, JUnit seems to have caught up with the times. Although the integration with build tools and IDEs still require some work, plugins for Maven and Gradle seem to be quite advanced, which seems to suggest that JUnit will continue to be the main Java testing framework for the foreseeable future.