BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Intro to .NET Unit & Integration Testing with SpecsFor

Intro to .NET Unit & Integration Testing with SpecsFor

 

SpecsFor is a highly-flexible framework for crafting both unit and integration tests for .NET applications. In this article, you'll learn everything you need to get up and running with SpecsFor, including selecting a test runner and how to create your first test project.

You will also see how SpecsFor eliminates common testing challenges, enabling you to focus on testing important logic and less on annoying setup.

Overview of SpecsFor

Hopefully the first question you’re asking is, “What exactly is SpecsFor?” It’s a testing framework designed to abstract away all the annoying testing concerns out of your way so that you can write clean tests quickly. It is both flexible and extensible.

It can be used for just about any kind of automated .NET testing need, including unit testing, integration testing, and full-blown end-to-end testing.

It’s probably easiest to understand SpecsFor if you see it in action. Here’s a spec for a MVC car factory, written using just the tools available with MS Test and a mocking framework:

[TestClass]
public class CarFactorySpecs
{
    [TestMethod]
    public void it_calls_the_engine_factory()
    {
        var engineFactoryMock = new Mock<IEngineFactory>();
        var sut = new CarFactory(engineFactoryMock.Object);
        sut.BuildMuscleCar();
        engineFactoryMock.Verify(x => x.GetEngine("V8"));
    }
    [TestMethod]
    public void it_creates_the_right_engine_type_when_making_a_car()
    {
        var engineFactoryMock = new Mock<IEngineFactory>();
        engineFactoryMock.Setup(x => x.GetEngine("V8"))
            .Returns(new Engine
            {
                Maker = "Acme",
                Type = "V8"
            });
        var sut = new CarFactory(engineFactoryMock.Object);
        var car = sut.BuildMuscleCar();
        Assert.AreEqual(car.Engine.Maker, "Acme");
        Assert.AreEqual(car.Engine.Type, "V8");
    }
}

There’s a lot of code here to test just a couple of things.

Now here’s the same spec, written with SpecsFor:

public class when_creating_a_muscle_car : SpecsFor<CarFactory>
{
    protected override void Given()
    {
        GetMockFor<IEngineFactory>()
            .Setup(x => x.GetEngine("V8"))
            .Returns(new Engine
            {
                Maker = "Acme",
                Type = "V8"
            });
    }
    private Car _car;
    protected override void When()
    {
        _car = SUT.BuildMuscleCar();
    }
    [Test]
    public void then_it_creates_a_car_with_an_eight_cylinder_engine()
    {
        _car.Engine.ShouldLookLike(() => new Engine
        {
            Maker = "Acme",
            Type = "V8"
        });
    }
    [Test]
    public void then_it_calls_the_engine_factory()
    {
        GetMockFor<IEngineFactory>()
            .Verify(x => x.GetEngine("V8"));
    }
}

What’s important here is what isn’t present:

  • There’s no repeated setup. In fact, there’s no code to create the factory at all, it’s handled by SpecsFor.
  • There’s no code to track mock objects. Again, SpecsFor does this for you.

There’s just a bit of setup and a couple of test cases.

We’ll examine how the SpecsFor version of this spec works later in this article, but for now, let’s take a peek at what makes SpecsFor work.

Built on the Shoulders of Giants

SpecsFor is built on top of a collection of great open-source tools. It pulls these tools together into a cohesive package that helps you overcome common testing hurdles quickly.

At its core, SpecsFor sits on top of NUnit, meaning any test runner or build server that supports NUnit will also work just fine with SpecsFor, no need for separate plug-ins or setup.

Next, SpecsFor provides Should, a library of extension methods for common test assertions. Instead of writing awkward to read assertions, like “Assert.AreEqual(x, 15),” you can write readable assertions like “x.ShouldEqual(15).” It’s a subtle change, but it makes a big impact!

Mocking is a topic that is heavily debated in the testing community, but mocking can be a useful tool. To that end, SpecsFor includes Moq, a library that makes it easy to create mock and stub objects for use in your tests.

Related to Moq, SpecsFor also includes an auto-mocking container, built on top of StructureMap. Using this container, SpecsFor can automatically create the class you wish to test along with all of its dependencies.

Finally, SpecsFor includes a solution to one of the most common (and noisy!) problems I’ve seen in .NET testing: comparing objects. The default equality implementation for objects will only return true if both X and Y point to the exact same instance in memory. So, the following spec will actually fail:

[Test]
public void equal_people_test()
{
    var person1 = new Person {Name = "Bob", Age = 29};
    var person2 = new Person {Name = "Bob", Age = 29};
    Assert.AreEqual(person1, person2);
}

The solution is to either to override the Equals method with your own, or compare each property on the objects. Both of these approaches tend to become painful to maintain.

By leveraging the awesome ExpectedObjects framework, the above spec can be rewritten as this:

[Test]
public void equal_people_test()
{
    var person1 = new Person {Name = "Bob", Age = 29};
    var person2 = new Person {Name = "Bob", Age = 29};
    person1.ShouldEqual(person2);
}

You can even do partial matching (checking only the subset of properties you care about). We’ll talk more on that later in this article! For now, let’s look at how to get started with SpecsFor.

Basic Usage

Thanks to tools such as NuGet, getting started with SpecsFor is actually very easy. Before you start though, you will need a test runner, something that can execute your test cases for you. Odds are you probably already have one (most .NET developers are doing some form of automated testing these days), but if you aren’t, here are a few that I recommend checking out.

  • ReSharper’s Test Runner – ReSharper is one of my favorite productivity tools, and it includes a test runner that’s capable of running NUnit tests, so it works just fine with specs you create using SpecsFor.
  • TestDriven.NET – This is one of the oldest (and most mature) .NET test runners out there. It can run tests created with just about any framework. Its UI is very minimal, which may or may not be your thing, but it’s worth checking out!
  • NCrunch – Whereas most test runners require you to kick off a test run manually, NCrunch is different: it automatically executes your tests in the background while you are writing code. It’s not exactly cheap, but it can be a huge time saver.
  • Visual Studio – Visual Studio Pro and up actually includes its own test runner. With the NUnit Adapter, it is able to execute NUnit test cases just fine. This is definitely my least-favorite test runner, but hey, it’s free!

Now let’s see how we can add SpecsFor to an existing solution. I’m going to be adding specs to a simple library that you can check out here on GitHub. You can follow along with that solution or one of you own. Assuming you already have the solution open in Visual Studio, go ahead and add a new Class Library (C#) project to your solution. This project will house our specs, and I usually follow a simple naming convention like “SolutionName.Specs.”

Your project will have a dummy Class1.cs file it. Go ahead and nuke that; we don’t need it!

While we’re at it, we also need to make sure our spec project references the project(s) we want to write tests for. Right-click on References again, and choose “Add Reference.” Choose “Solution” and then check the applicable project(s).

(Click on the image to enlarge it)

Now we just need to add SpecsFor to the project. Right-click on references and choose “Manage NuGet Packages.” This will open the NuGet package manager. Search online for “SpecsFor,” find the “SpecsFor” package (ignore the related packages for now), and install it!

(Click on the image to enlarge it)

When you install SpecsFor, it will automatically install all the other libraries that SpecsFor uses to make your life easier, so you’ll now have SpecsFor, NUnit, Moq, Should, ExpectedObjects, and StructureMap.AutoMocking!

Now we’re ready to create our first spec. If you’re following along with the sample project, our domain includes a Car object. Let’s write some specs to see what happens when a running car is stopped.

NOTE: We’ll be following the naming and organization conventions I like to use, but SpecsFor doesn’t care about how you name or organize your specs. Do whatever feels best to you!

Create a new class in your spec project called CarSpecs. In our new class, let’s also go ahead and add in the namespaces we need: NUnit.Framework, Should, SpecsFor, and the namespace for the class we want to test (for the sample project that will be InfoQSample.Domain). Your class should now look something like this:

using NUnit.Framework;
using Should;
using SpecsFor;
using InfoQSample.Domain;
namespace InfoQSample.Specs
{
    public class CarSpecs
    {
    }
}

Now we’re ready to write our spec!

SpecsFor follows the Given-When-Then, BDD language for specifying your test cases. This language consists of three phases:

  • Given – Establish your state. What is true before your spec executes?
  • When: Perform an action. Call the method(s) you are testing on your target.
  • Then: Verify. Make sure the world looks the way you expect it to look.

Using this same language, here’s the spec we’re going to implement in English:

Given a car is running
When a car is stopped
Then the car stops
Then the engine stops

Specs in SpecsFor all derive from the base SpecsFor<T> class, where T is the type you wish to write test cases against. In our case, T will be Car. I like to name my specs after the scenario I’m testing, so here’s what I came up with:

public class CarSpecs
{
    public class when_a_car_is_stopped : SpecsFor<Car>
    {
    }
}

NOTE: Again, I like to nest each scenario within my larger spec class, but this organization is completely optional.

It doesn’t look like much, but the simple act of deriving from SpecsFor<Car> will automatically give us an instance of Car to write our test cases against. We don’t have to create it ourselves or manage its lifecycle. SpecsFor provides a property, SUT (System Under Test), that will be populated automatically with an instance of Car for us.

Now let’s walk through and implement our spec. First, we need to establish our state: “Given a car is running…

protected override void Given()
{
    //Given a car is running (start the car!)
    SUT.Start();
}

Notice that SpecsFor provides a virtual Given method that we can override to establish our state. SpecsFor will call this method for us exactly once, ensuring the universe exists in the state we expect. For our spec, this code just starts our car, so now we have a running car to work with.

Now we can perform our action, “When a car is stopped…”

protected override void When()
{
    //When a car is stopped
    SUT.Stop();
}

Again, SpecsFor provides a virtual When method that we can hook in to. Now our car is stopped! So what should the world look like now? “Then the car stops…”

[Test]
public void then_the_car_is_stopped()
{
    SUT.IsStopped.ShouldBeTrue();
} 

There is no base method to override here since your spec will typically need to verify multiple things. Instead, we just add a public method and decorate it with NUnit’s Test attribute. Notice how we’re asserting that the world looks the way we expect here: using a fluent extension method from the Should library.

Our spec actually specified two things that should be true, so let’s add a second test case:

[Test]
public void then_the_engine_is_stopped()
{
    SUT.Engine.IsStopped.ShouldBeTrue();
} 

Again, we add a public method, decorate it with the Test attribute, then use Should’s fluent extension methods to make sure everything looks the way we expect.

Using your test runner of choice, now you can run your spec, and you should see everything passing!

Our full spec now looks like this:

using Should;
using SpecsFor;
using InfoQSample.Domain;
namespace InfoQSample.Specs
{
    public class CarSpecs
    {
        public class when_a_car_is_stopped : SpecsFor<Car>
        {
            protected override void Given()
            {
                //Given a car is running (start the car!)
                SUT.Start();
            }
            protected override void When()
            {
                //When a car is stopped
                SUT.Stop();
            }
            [Test]
            public void then_the_car_is_stopped()
            {
                SUT.IsStopped.ShouldBeTrue();
            }
            [Test]
            public void then_the_engine_is_stopped()
            {
                SUT.Engine.IsStopped.ShouldBeTrue();
            }
        }
    } 

} 

I want to call your attention to what isn’t in this spec again: there’s no code to actually create our Car. It’s done for us automatically. We don’t have to start and stop our car in each test case, because we were able to use SpecsFor’s Given and When methods to encapsulate those operations. We’re left with a spec that contains just the interesting bits, and no more.

This is, admittedly, a trivial spec. Even without SpecsFor, there wouldn’t be much to this one. Let’s see how things look if we tackle a more complex class.

Mocks and Auto-Mocking

One of the big advantages of SpecsFor is that you don’t have to deal with the dependencies of your System Under Test. It takes care of them for you. In the case of abstract or interface dependencies, SpecsFor will automatically create mock objects for you, keep up with them, and make them available to both your spec and your System Under Test. Let’s see how we can leverage this to write a spec for our sample project’s CarFactory.

First, let’s look at our spec in English:

When creating a muscle car
Then it creates a car with an eight cylinder engine
Then it gets the engine from the engine factory

This spec requires a little more set up. If we look at our CarFactory, we can see that it has a dependency on a type that implements the IEngineFactory interface:

public class CarFactory
{
    private readonly IEngineFactory _engineFactory;
    public CarFactory(IEngineFactory engineFactory)
    {
        _engineFactory = engineFactory;
    }
    public Car BuildMuscleCar()
    {
        return new Car(_engineFactory.GetEngine("V8"));
    }
} 

SpecsFor will automatically create a mock object (using Moq) for IEngineFactory for us, and it will supply this mock to our factory when it initializes the SUT property. We just need to configure the mock to behave the way we want it to. A full discussion of using Moq objects is beyond the scope of this article, but you can check out the Moq documentation for a thorough set of examples.

For our spec, we just need to tell our mock IEngineFactory to return an engine. We can do that by first asking SpecsFor to give us the mock object, then we can configure it. We’ll do this in our spec’s Given method:

protected override void Given()
{
    GetMockFor<IEngineFactory>()
        .Setup(x => x.GetEngine("V8"))
        .Returns(new Engine
        {
            Maker = "Acme",
            Type = "V8"
        });
} 

We use the GetMockFor<T> method to get a mock by its type, then we can use Moq’s API to configure it. In this case, we’re telling the mock, “When you are asked for a V8 engine, hand back this V8 engine made by Acme.”

Now we can continue on with our spec, implementing the When method:

private Car _car;
protected override void When()
{
    _car = SUT.BuildMuscleCar();
}

We call the BuildMuscleCar method, which returns an instance of Car back to us. Because our test cases will need to operate on this car, we capture it in a field.

Now we can make sure the car has the expected engine. We could try to do something like this:

[Test]
public void then_it_creates_a_car_with_an_eight_cylinder_engine()
{
    _car.Engine.ShouldEqual(new Engine
    {
        Maker = "Acme",
        Type = "V8"
    });
}

Unfortunately though, our spec would fail. Remember, by default, two objects are only equal if they point to the exact same instance in memory!

Instead of checking equality, we’ll use SpecsFor’s ShouldLookLike extension method, which allows us to check whether two objects look the same, even if they don’t point to the exact same instance in memory:

[Test]
public void then_it_creates_a_car_with_an_eight_cylinder_engine()
{
    _car.Engine.ShouldLookLike(() => new Engine
    {
        Maker = "Acme",
        Type = "V8"
    });
}

Using this version of ShouldLookLike, only properties we specify will be checked. Any other properties on the engine will be ignored.

Finally, let’s also make sure that our CarFactory actually called the engine factory. We can do that by again asking SpecsFor to hand us the mock object, then using Moq’s API to verify that the GetEngine method was called.

[Test]
public void then_it_calls_the_engine_factory()
{
    GetMockFor<IEngineFactory>()
        .Verify(x => x.GetEngine("V8"));
}

 

This spec is a bit redundant since we are already making sure that the correct type of engine is being added to the car, but it does illustrate how we can not only configure mocks, but also verify interactions against them.

We’re still just scratching the surface of SpecsFor. Let’s look at some of the ways you can extend its behavior and bend it to your will.

Advanced SpecsFor Topics

Just about everything in SpecsFor can be overridden so that you can add your own behavior. For example, you may not always want SpecsFor to create the System Under Test for you. There may be times when you need to create it yourself instead. You can override the InitializeClassUnderTest method for these situations:

protected override void InitializeClassUnderTest()
{
    SUT = new CarFactory(new RealEngineFactory());
}

You can also reconfigure the auto-mocking container that SpecsFor uses. This is useful when you want to use a real, concrete type in place of a mock object for an interface:

protected override void ConfigureContainer(IContainer container)
{
    container.Configure(cfg =>
    {
        cfg.For<IEngineFactory>().Use<RealEngineFactory>();
    });
}

You can also use the ConfigureContainer method to load in entire application registries (if you are using StructureMap), which can be very useful if you are doing integration testing.

SpecsFor also includes a configurable system that allows you to attach behaviors to your specs based on conventions. By creating a NUnit SetUpFixture that derives from the SpecsForConfiguration class, you can define your own conventions. You can do some powerful things with this system, such as creating transactional, isolated specs that use Entity Framework. Here’s a simple example:

//The configuration class...
[SetUpFixture]
public class SpecsForConfig : SpecsForConfiguration
{
    public SpecsForConfig()
    {
        WhenTesting<INeedDatabase>().EnrichWith<EFDatabaseCreator>();
        WhenTesting<INeedDatabase>().EnrichWith<TransactionScopeWrapper>();
        WhenTesting<INeedDatabase>().EnrichWith<EFContextFactory>();
    }
}
//The "marker" interface...
public interface INeedDatabase : ISpecs
{
    AppDbContext Database { get; set; }
}
//The class that creates the EF database for your specs to use
public class EFDatabaseCreator : Behavior<INeedDatabase>
{
    private static bool _isInitialized;
    public override void SpecInit(INeedDatabase instance)
    {
        if (_isInitialized) return;
                               Directory.GetCurrentDirectory());
        var strategy = new MigrateDatabaseToLatestVersion
            <AppDbContext, Configuration>();
        Database.SetInitializer(strategy);
        using (var context = new AppDbContext())
        {
            context.Database.Initialize(force: true);
        }
        _isInitialized = true;
    }
}
//The class that ensures all your operations are wrapped in a transaction that's
//rolled back after each spec executes.
public class TransactionScopeWrapper : Behavior<INeedDatabase>
{
    private TransactionScope _scope;
    public override void SpecInit(INeedDatabase instance)
    {
        _scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled);
    }
    public override void AfterSpec(INeedDatabase instance)
    {
        _scope.Dispose();
    }
}
//And the factory that provides an instance of your EF context to your specs
public class EFContextFactory : Behavior<INeedDatabase>
{
    public override void SpecInit(INeedDatabase instance)
    {
        instance.Database = new AppDbContext();
        instance.MockContainer.Configure(cfg =>
                cfg.For<AppDbContext>().Use(instance.Database));
    }
    public override void AfterSpec(INeedDatabase instance)
    {
        instance.Database.Dispose();
    }
}

With these conventions in place, you can then easily write specs that fully-exercise your application logic all the way down to the database, like in this example for an MVC controller that lists tasks:

public class when_getting_a_list_of_tasks 
    : SpecsFor<HomeController>, INeedDatabase
{
    public AppDbContext Database { get; set; }
    protected override void Given()
    {
        for (var i = 0; i < 5; i++)
        {
            Database.Tasks.Add(new Task
            {
                Id = Guid.NewGuid(),
                Title = "Task " + i,
                Description = "Dummy task " + i
            });
        }
        Database.SaveChanges();
    }
    private ActionResult _result;
    protected override void When()
    {
        _result = SUT.Index();
    }
    [Test]
    public void then_it_returns_tasks()
    {
        _result.ShouldRenderDefaultView()
            .WithModelType<TaskSummaryViewModel[]>()
            .Length.ShouldEqual(5);
    }
}

This example also illustrates another aspect of SpecsFor we haven’t touched on yet: the SpecsFor<Web> Helpers library.

SpecsFor<Web> Helpers

One of the oft-touted benefits of ASP.NET MVC over traditional ASP.NET WebForms is testability. However, if you’ve spent much time writing specs against an ASP.NET MVC application, you will probably agree that the testability story still leaves quite a lot to be desired, especially if you want to test anything more than simple controller actions. Testing action filters and HTML helpers requires the creation of extensive mock or fake objects, with things wired together in just the right way. If you fail to wire the pieces together perfectly, you’ll be met by NullReferenceExceptions and other land mines.

That’s where the SpecsFor<Web> Helpers library comes in. This add-on library for SpecsFor aims to help you overcome many of these common MVC testing hurdles.

The saying goes, “a picture is worth a thousand words,” but I believe the same can be said for code snippets. Here’s an example of testing that an action returns the expected view without SpecsFor<Web> Helpers:

[Test]
public void then_it_says_hello_to_the_user()
{
    var viewResult = _result.ShouldBeType<ViewResult>();
    var model = viewResult.Model.ShouldBeType<SayHelloViewModel>();
    model.ShouldLookLike(new SayHelloViewModel
    {
        Name = "John Doe"
    });
}

And here’s the same spec, cleaned up using SpecsFor<Web> Helpers:

[Test]
public void then_it_says_hello_to_the_user()
{
    _result.ShouldRenderDefaultView()
      .WithModelLike(new SayHelloViewModel
      {
          Name = "John Doe"
      });
}

Testing a redirect is a horrid mass of strings without SpecsFor<Web> Helpers:

[Test]
public void then_it_redirects_to_the_say_hello_action()
{
    var redirectResult = _result.ShouldBeType<RedirectToRouteResult>();
    redirectResult.RouteValues["controller"].ShouldEqual("Home");
    redirectResult.RouteValues["action"].ShouldEqual("SayHello");
    redirectResult.RouteValues["name"].ShouldEqual("Jane Doe");
}

It’s quite a bit simpler once you have SpecsFor<Web> Helpers in the mix:

[Test]
public void then_it_redirects_to_the_say_hello_action()
{
    _result.ShouldRedirectTo<HomeController>(
      c => c.SayHello("Jane Doe"));
}

Want to test an action filter? Good luck! You’ll need to dummy up an ActionExecutingContext to use. Here’s the code it takes, discovered through much pain and suffering:

private ActionExecutingContext _filterContext;
protected override void When()
{
    var httpContext = new Mock<HttpContextBase>().Object;
    var controllerContext = new ControllerContext(httpContext, new RouteData(), new Mock<ControllerBase>().Object);
    var reflectedActionDescriptor = new ReflectedActionDescriptor(typeof(ControllerBase).GetMethods()[0], "Test", new ReflectedControllerDescriptor(typeof(ControllerBase)));
    _filterContext = new ActionExecutingContext(controllerContext, reflectedActionDescriptor, new Dictionary<string, object>());
    SUT.OnActionExecuting(_filterContext);
} 

But with SpecsFor<Web> Helpers, you can just pass in the pre-made FakeActionExecutingContext instead:

private FakeActionExecutingContext _filterContext;
protected override void When()
{
    _filterContext = new FakeActionExecutingContext();
    SUT.OnActionExecuting(_filterContext);
}

There’s a lot more in the SpecsFor<Web> Helpers library (too much to cover in this article), so check out the docs for more examples.

Next Steps

If nothing else, I hope you’re at least curious about SpecsFor now. If you want to try it out, getting started is as easy as picking a test runner and adding the NuGet package to a new class library project. And if you are doing ASP.NET MVC development, you can make your tests easier to write (and read!) by leveraging the SpecsFor<Web> Helpers library, also available via NuGet.

If you want to learn more, there are several resources to help you out. The official docs contain additional samples and provide solutions to common testing challenges. For those that prefer structured instruction, check out my Pluralsight course on SpecsFor. And if you learn best by digging through source code, then head over to the GitHub repo where all the code for SpecsFor and its related tools are housed.

About the Author

Matt Honeycutt is a software architect specializing in ASP.NET web applications, particularly ASP.NET MVC.  He’s an avid practitioner of Test-Driven Development, creating both the SpecsFor and SpecsFor.Mvc frameworks. He has served as the lead developer on numerous multi-million dollar software projects and enjoys finding elegant solutions to difficult problems. He holds a Masters of Science in Computer Science, he authors courses for Pluralsight.com, and he’s published papers in research journals and conferences on topics ranging from data mining and machine learning to human-computer interaction. He blogs at trycatchfail.com, and he is a frequent speaker at software conferences in Tennessee.

Rate this Article

Adoption
Style

BT