xUnit.net - Next Generation of Unit Testing Frameworks?

Jim Newkirk, creator of NUnit, has announced a new Unit Testing Framework called xUnit.net. The proclaimed successor to NUnit is supposed to get rid of NUnit's mistakes and shortcomings and add some best practices and extensibility to the framework.

Jim Newkirk and Brad Wilson, the creators of xUnit.net, have identified the following bad practices, shortcomings and improvements from the experience with NUnit and other Unit Testing frameworks:

• Single Object Instance per Test Method.
• No [SetUp] or [TearDown].
• No [ExpectedException].
• Aspect-Like Functionality.
• Reducing the Number of Custom Attributes.
• Use of Generics.
• Anonymous Delegates.
• Assert extensibility.
• Test method extensibility.
• Test class extensibility.

xUnit.net reduces the amount of .NET Attributes, which control the tests and their execution. There is a single Attribute [Test], which marks a test method. Unlike NUnit, MbUnit, or MSTest, Test classes are not marked. XUnit simply looks for all public test methods in all public classes within the assembly. [SetUp] and [TearDown] methods have been abandoned as being "generally bad" practice:

The xUnit.net team feels that per-test setup and teardown creates difficult-to-follow and debug testing code, often causing unnecessary code to run before every single test is run.

Jim Newkirk has written about "Why you should not use SetUp or TearDown in NUnit on his blog:

The problem that I have with SetUp in this case is twofold. The first and primary complaint is that when I am reading each test I have to glance up to BeforeTest() to see the values that are being used in the test. Worse yet if there was a TearDown method I would need to look in 3 methods. The second issue is that BeforeTest() initializes member variables for all 3 tests which complicates BeforeTest() and makes it violate the single responsibility pattern.

The [ExpectedException] Attribute, which expects the declared Exception to be thrown by the test code, has been replaced by the Assert.Throws assertion. TestFixtures are introduced by an ITestFixture interface, which declares two methods: BeforeAllTests() and AfterAllTests(). Test timeouts and temporarily skipping a test are implemented by parameters of the [Test] attribute instead of full fledged attributes. The very popular [RowTest] and [Row] test patterns of MbUnit have been included as [Theory] and [DataViaXxx]:

The xunit.extensions.dll ships with support for data-driven tests call Theories. Mark you test with [Theory] (instead of [Test]), and then mark it with one of the [DataVia...] attributes to indicate where the data will come from.

Assertions have also been reduced in xUnit.net. Every assertion, whose functionality can be achieved by one of the basics, has been left out. In addition the prefixes "is" and "are", e.g. "AreEqual" or "IsEmpty", have been removed. The xUnit.net site provides an extensive comparison of NUnit, MbUnit, MSTest, and xUnit.net attributes and assertions.

xUnit.net also makes use of the new language features of .NET 2.0 and 3.5. It supports the use of generics, which for instance provide type-safety in comparers, such as Equal and NotEqual assertions. The Assert.Throws() method, which replaces the [ExpectedException] attribute, expects anonymous delegates or lambda expressions, which encapsulate the functionality, which is expected to throw the declared exception. The code gets more compact and more readable:

Assert.Throws(delegate { operation(); }); // .NET 2.0
Assert.Throws(() => operation()); // .NET 3.5

Test classes, test methods, and assertions can easily be extended. The IComparer interface allows to extend the functionality of Equals, NotEqual and suchlike. xUnit.net supports the creation of test patterns, which control how tests are invoked and executed. Finally the execution of a test suite or test class can be controlled by extending existing or creating new test runners.

The creators of xUnit.net clearly think of their open source framework as the successor to NUnit. Roy Osherove thinks that xUnit.net is still premature and has some doubts about its future.