Behavioral Testing

Jimmy Bogard recently gave a presentation at the NDCOslo about a testing strategy he calls Holistic Testing. Have a look at the video, it’s worth your time!

Among a lot of other things he talked about why he doesn’t use mocking frameworks (or hand-crafted mock objects) very often in his tests. According to Jimmy, testing with mocks tends to couple your tests to the implementation details of your code. Instead he prefers to test the behavior of his code and ignore said implementation as far as possible.

While I don’t agree with his sample code I totally agree with his statement. But let’s have a look.

[Theory, AutoData]
public void DummyTest(Customer customer)
{
  var calculator = A.Fake<ITaxCalculator>();
  A.CallTo(() => calculator.Calculate(customer)).Returns(10);
  var factory = new OrderFactory(calculator);
  var order = factory.Build(customer);
  order.Tax.ShouldEqual(10);
}

This code snippet uses xUnit’s Theories for data driven tests along with AutoFixture’s extension to that feature which “deterministically creates random test data”. AutoFixture creates the Customer object. Then Jimmy uses FakeItEasy to create a mock for the ITaxCalculator, setup the return value for the call to it’s Calculate() method. Passes the calculator to the constructor of the OrderFactory lets the factory create an Order and asserts that the correct tax is set on the order. Nothing spectacular so far (if you are already familiar with data Theories and AutoFixture that is…).

But what happens if the factory no longer uses a calculator? Or if we add another parameter to the constructor? After all the constructor is just an implementation detail that we don’t want to care about. But these changes will break our test code. And this is where Jimmy gets fancy.

Assuming that we already build our code according to the Dependency Injection Pattern and use a DI container in our project (that’s quite some assumption…), we already have the information about how the factory is assembled and which calculator (if any) is to be used in the configuration of said container. So why not use the container to provide us with the factory instead of new’ing it up ourselves?

I took the liberty of streamlining Jimmy’s test a bit more by letting AutoFixture create the customer as well. The ContainerDataAttribute is derived from AutoFixture’s AutoDataAttribute. My sample uses Unity instead of StructureMap but it works all the same. In contradiction to Jimmy’s statement (around 45:46 in the video) Unity does support the creation child containers. Although I have to admit that this is the first scenario where this feature makes sense to me. But that is a different story I’ll save for another day.

[Theory, ContainerData]
public void DummyTest(OrderFactory factory, Customer customer)
{
  Order order = factory.Build(customer);
  order.Tax.ShouldEqual(10);
}

Wow. That’s what I call straight to the point. You don’t care about the creation process of your system-under-test (the factory) or your test data (the customer). You just care about the behavior which is to set the correct tax rate on your order object.
As the tax rate differs from one state to another it might not make sense to test for that specific value without controlling its calculation to some extent (e.g. via a mock object…) but that’s a common problem with HelloWorld!-samples and does not diminish the brilliancy of the general concept.

So let’s see how the ContainerDataAttribute is implemented.

public class ContainerDataAttribute : AutoDataAttribute
{
  public ContainerDataAttribute()
    : base(new Fixture().Customize(
      new ContainerCustomization(
        new UnityContainer().AddExtension(
          new ContainerConfiguration()))))
  {
  }
}

We derive from AutoFixture’s AutoDataAttribute which does the heavy lifting for us (like integrating with xUnit, creating random test data, …). We call the base class’ constructor handing it a customized Fixture object (that is AutoFixture’s lynchpin for creating test data). The customization receives a preconfigured UnityContainer instance as a parameter. As you might have guessed UnityContainer is the Unity DI container. Unity’s configuration system is not as advanced as that of most other containers. Especially it does not bring a dedicated way to package configuration data (like StructureMap’s Registry for example). But you can (ab)use the UnityContainerExtension class to achieve the same result. Just place your configuration code inside your implementation of the abstract Initialize() method and add the extension to the container.

public class ContainerConfiguration : UnityContainerExtension
{
  protected override void Initialize()
  {
    this.Container.RegisterType<IFoo, Foo>();
    this.Container.RegisterType<ITaxCalculator, DefaultTaxCalculator>();
  }
}

It makes sense to re-use as much of your production configuration as possible. But you should consider to modify it in places where the tests might interfere with actual production systems (like sending emails, modifying production databases etc.).

The customization to AutoFixture hooks up two last-chance handlers for creating objects by adding them to IFixture.ResidueCollectors.

public class ContainerCustomization : ICustomization
{
  private readonly IUnityContainer container;
  public ContainerCustomization(IUnityContainer container)
  {
    this.container = container;
  }
  public void Customize(IFixture fixture)
  {
    fixture.ResidueCollectors.Add(new ChildContainerSpecimenBuilder(this.container));
    fixture.ResidueCollectors.Add(new ContainerSpecimenBuilder(this.container));
  }
}

AutoFixture calls these object creators SpecimenBuilders. The first one we hook up is responsible for creating a child container if a test requires an IUnityContainer as a method parameter. The second actually uses the container to create objects.

public class ChildContainerSpecimenBuilder : ISpecimenBuilder
{
  private readonly IUnityContainer container;
  public ChildContainerSpecimenBuilder(IUnityContainer container)
  {
    this.container = container;
  }
  public object Create(object request, ISpecimenContext context)
  {
    Type type = request as Type;
    if (type == null || type != typeof(IUnityContainer))
    {
      return new NoSpecimen();
    }
    return this.container.CreateChildContainer();
  }
}
public class ContainerSpecimenBuilder : ISpecimenBuilder
{
  private readonly IUnityContainer container;
  public ContainerSpecimenBuilder(IUnityContainer container)
  {
    this.container = container;
  }
  public object Create(object request, ISpecimenContext context)
  {
    Type type = request as Type;
    if (type == null)
    {
      return new NoSpecimen();
    }
    return this.container.Resolve(type);
  }
}

The NoSpecimen class is AutoFixture’s way of telling its kernel that this builder can’t construct an object for the current request. Something like a NullObject.

Well and that’s it. A couple of dozen lines of code. A superior testing framework (xUnit). A convenient way of creating test data (AutoFixture). A DI container (which should be mandatory for any project if you ask me…). And writing maintainable tests for the correct behavior of your code becomes a piece of cake. I love it! 🙂

You can find the sample code on my playground on CodePlex (solution TecX.Playground, project TecX.BehavioralTesting).

Advertisements

New Release: Enterprise Library 6.0 & Unity 3.0

Patterns & practices (p&p) just released a new version of the Enterprise Library and the Unity Dependency Injection Container.

Grigori’s release notes can be found here.

The binaries and source code for EntLib can be downloaded from MSDN. Those for Unity are quite well hidden for some reason… Grab them here.

Microsoft p&p Advisory Board Member

Last week Grigori Melnik, Principal Program Manager at Microsoft patterns & practices and responsible for the development of the Microsoft Enterprise Library and Unity Dependency Injection Container, asked wether I would like to join the Advisory Board for EntLib6/Unity3.

I happily accepted the invitation 🙂

I’m looking forward to participating in the development of a great tool set and learning how p&p works! Thanks for the chance to give something back to you guys!

 

 

 

(Time-)Tracing with Unity Interception, Decorators and Reflection

We use tracing to find out what takes us so long when processing X or computing Y. Tracing is a cross-cutting concern. It can be used anywhere in our applications. As such there are some well-known ways to implement it to keep our business code clean such as using hand-crafted decorators or an interception framework of our preference.

This is a non-scientific comparison of three different approaches. One uses Unity’s Interception Extension, two use decorators where one figures out what method it is currently tracing using reflection and the other needs magic strings for the same purpose.

There are many ways to measure the time it takes to perform a certain operation. Below you see the easiest and most convenient way I found over time.

public class TraceTimer : IDisposable
{
  private readonly string operationName;
  private readonly DateTime start = DateTime.Now;
  public TraceTimer()
  {
    StackTrace trace = new StackTrace();
    MethodBase method = trace.GetFrame(1).GetMethod();
    this.operationName = MethodSignatureHelper.GetMethodSignature(method);
  }
  public TraceTimer(string operationName)
  {
    this.operationName = operationName;
  }
  public void Dispose()
  {
    TimeSpan elapsed = DateTime.Now - this.start;
    string msg = string.Format("{0} took {1:F3}s", this.operationName, elapsed.TotalSeconds);
    Debug.WriteLine(msg);
  }
}

And this is how it is used:

using (new TraceTimer("Foo()"))
{
  Foo();
}

Which writes a message like ‘Foo() took 1.038s‘  to the debug console.

Using System.Diagnostics.Debug for writing your measurements is most often not the way you want to go in a real application. You can use a logging framework of your choice instead. But Debug does the job for our purpose here.

The TraceTimer has two constructors. One takes the name of the operation it measures as a parameter. The other extracts the method signature by using a StackFrame.

Unity Interception

Unity uses classes that implement IInterceptionBehavior to intercept calls to a certain target. For this test the behavior looks like this:

public class DummyTracingBehavior : IInterceptionBehavior
{
  public IMethodReturn Invoke(IMethodInvocation input, GetNextInterceptionBehaviorDelegate getNext)
  {
    var methodSignature = MethodSignatureHelper.GetMethodSignature(input.MethodBase);
    using (new TraceTimer(methodSignature))
    {
      return getNext()(input, getNext);
    }
  }
  public IEnumerable<Type> GetRequiredInterfaces() { return Type.EmptyTypes; }
  public bool WillExecute { get { return true; } }
}

And this is the test setup:

var container = new UnityContainer();
container.AddNewExtension<Interception>();
container.RegisterType<IDummy, Dummy>(
  new Interceptor<InterfaceInterceptor>(),
  new InterceptionBehavior<DummyTracingBehavior>());
using (new TraceTimer("WithInterception"))
{
  for (int i = 0; i < NumberOfRunsPerCycle; i++)
  {
    var dummy = container.Resolve<IDummy>();
    dummy.DoIt(i);
  }
}

Hand-crafted Decorator

The decorator implements the same interface as the target and delegates all calls to that target.

public class DummyDecorator : IDummy
{
  private readonly IDummy inner;
  public DummyDecorator(IDummy inner)
  {
    this.inner = inner;
  }
  public string DoIt(int i)
  {
    using (new TraceTimer("DoIt(int)"))
    {
      return this.inner.DoIt(i);
    }
  }
}

It won’t get much simpler than that. And again the test setup:

var container = new UnityContainer();
container.RegisterType<IDummy, Dummy>("Dummy");
container.RegisterType<IDummy, DummyDecorator>(
  new InjectionConstructor(new ResolvedParameter<IDummy>("Dummy")));
using (new TraceTimer("WithDecorator"))
{
  for (int i = 0; i < NumberOfRunsPerCycle; i++)
  {
    var dummy = container.Resolve<IDummy>();
    dummy.DoIt(i);
  }
}

Decorator with Reflection

The second decorator does not rely on magic strings to identify the method name and parameters but uses reflection to figure those details out on its own.

public class DummyDecoratorWithReflection : IDummy
{
  private readonly IDummy inner;
  public DummyDecoratorWithReflection(IDummy inner)
  {
    this.inner = inner;
  }
  public string DoIt(int i)
  {
    using (new TraceTimer())
    {
      return this._inner.DoIt(i);
    }
  }
}

The only thing that differs between the two decorators is the missing string parameter in the constructor call of the TraceTimer. The test setup is identical. Just replace the type DummyDecorator with DummyDecoratorWithReflection.

Prerequisites

For each of the three approaches I set the NumberOfRunsPerCycle to 10,000 and ran 10 cycles for each test.

Each test uses Unity to create a new test object for each run. That is by intention. To work with interception you need Unity anyway. The decorators can be new’ed up which saves you something between 2 and 5% per run. But you will loose Unity’s support for auto-wiring your targets. I prefer convenience and configurability over the last ounce of performance as long as there is no business need for that performance. So whenever possible I use a container to create my objects.

Results

I added up the times measured for each cycle and calculated the average time per cycle. The ranking is as follows:

  1. Hand-crafted Decorator with 3.1015s
  2. Unity Interception with 3.4265s
  3. Decorator with Reflection with 3.5391

The total time taken is not really important as it depends on the machine that runs the tests, the logging framework you use to write your measurements, wether you run the tests as console application or unit-test etc. pp.

But it allows you to compare how one solution performs compared to the others. Not surprisingly the hand-crafted decorator is the fastest solution. But it is also the one with the lowest convenience for your developers. For each target (interface) you want to trace you would have to implement a decorator and you would have to provide the method signature on your own. If that signature changes you would have to change your decorator’s code as well.

Unity’s interception mechanism is about 10% slower than hand-crafted code. But it’s convenience factor is way higher. You write one implementation and all you have to do then is to change your configuration to enable it for arbitrary targets.

Looking at the numbers decorators that use reflection to figure out what is happening seem more like an academic exercise. They are slower than interception but provide less convenience. You have to write a decorator for each target but you don’t get the performance of the first solution. Yet you don’t have to care about changes to the method’s signature. If you can’t use Unity for one reason or another but want to avoid the overhead of maintaining your decorators they might still be a choice. Otherwise I think they are a suboptimal solution.

Enum support for Unity

Today I stumbled upon a question on StackOverflow that was related to the resolution of an enum-Type dependency using StructureMap.

And I thought to myself: Can Unity handle enums? Turns out it can’t. But yet again the flexible architecture allows you to retrofit that feature with ease.

First we need a strategy that detects when we are trying to resolve an enum-Type:

public class EnumStrategy : BuilderStrategy
{
  public override void PreBuildUp(IBuilderContext context)
  {
    if (context.BuildKey.Type.IsEnum && context.Existing == null)
    {
      IBuildPlanPolicy policy = new EnumPolicy(context.BuildKey.Type);
      context.Policies.Set(policy, context.BuildKey);
    }
  }
}

If the requested Type is an enum we place an IBuildPlanPolicy in the policy list that knows how to create the default value of that enum-Type.

public class EnumPolicy : IBuildPlanPolicy
{
  private readonly Type enumType;
  public EnumPolicy(Type enumType)
  {
    this.enumType = enumType;
  }
  public void BuildUp(IBuilderContext context)
  {
    if (context.Existing == null)
    {
      context.Existing = Enum.ToObject(this.enumType, 0);
    }
  }
}

We show some manners and make sure that there is no value inside the BuilderContext before we fill in the enum value.
And finally we need an extension that installs the strategy in the pipeline:

public class EnumExtension : UnityContainerExtension
{
  protected override void Initialize()
  {
    var strategy = new EnumStrategy();
    this.Context.Strategies.Add(strategy, UnityBuildStage.PreCreation);
  }
}

Where another test proves that Unity’s default behavior is to throw a ResolutionFailedException when it is asked to resolve an enum-Type dependency now the following test runs green:

[Fact]
public void CanResolveDefaultEnumValueForByteBasedEnum()
{
  var container = new UnityContainer().AddNewExtension();
  var sut = container.Resolve<DependsOnEnumBasedOnByte>();
  Assert.Equal(EnumBasedOnByte.Default, sut.Enum);
}

As the names of the test classes indicate this also works for enum-Types that are not derived from integer. Problem solved.

Get the source code for the EnumExtension  here (project TecX.Unity folder Enums and the test suite that shows how to use it in TecX.Unity.Test).

Register multiple instances without naming them

The wishlist for Unity vNext contains an item with the following description:

Register multiple instances without naming them and then resolve all these instances by ResolveAll

Well that does not sound that difficult does it? You just need to mess a little with Unity’s type mapping system.

Lets write a test first that will show we got it right:

[TestMethod]
public void CanResolveMultipeDefaultMappingsUsingResolveAll()
{
  var container = new UnityContainer().AddNewExtension<Remember>();
  container.RegisterType<IFoo, One>();
  container.RegisterType<IFoo, Two>();
  container.RegisterType<IFoo, Three>();
  IFoo[] foos = container.ResolveAll<IFoo>().OrderBy(f => f.GetType().Name).ToArray();
  Assert.AreEqual(3, foos.Length);
  Assert.IsInstanceOfType(foos[0], typeof(One));
  Assert.IsInstanceOfType(foos[1], typeof(Three));
  Assert.IsInstanceOfType(foos[2], typeof(Two));
}

Armed with the Unity source code I started digging for the truth.

The call to ResolveAll uses an internal class called NamedTypeRegistry to find out which mappings where registered as named mappings. You can’t access this class directly but the ExtensionContext that is provided to every class derived from UnityContainerExtension can add mappings to the registry. The code for the Remember extension does just that. Add a mapping to the registry and add another BuildKeyMappingPolicy that maps the named mapping to one of the default mappings.

public class Remember : UnityContainerExtension
{
  protected override void Initialize()
  {
    this.Context.Registering += this.OnRegistering;
    this.Context.RegisteringInstance += this.OnRegisteringInstance;
  }
  private void OnRegisteringInstance(object sender, RegisterInstanceEventArgs e)
  {
    if(string.IsNullOrEmpty(e.Name))
    {
      string uniqueName = Guid.NewGuid().ToString();
      this.Context.RegisterNamedType(e.RegisteredType, uniqueName);
      this.Context.Policies.Set<IBuildKeyMappingPolicy>(
        new BuildKeyMappingPolicy(new NamedTypeBuildKey(e.RegisteredType)),
        new NamedTypeBuildKey(e.RegisteredType, uniqueName));
    }
  }
  private void OnRegistering(object sender, RegisterEventArgs e)
  {
    if (e.TypeFrom != null && string.IsNullOrEmpty(e.Name))
    {
      string uniqueName = Guid.NewGuid().ToString();
      this.Context.RegisterNamedType(e.TypeFrom, uniqueName);
      if (e.TypeFrom.IsGenericTypeDefinition && e.TypeTo.IsGenericTypeDefinition)
      {
        this.Context.Policies.Set<IBuildKeyMappingPolicy>(
          new GenericTypeBuildKeyMappingPolicy(new NamedTypeBuildKey(e.TypeTo)),
          new NamedTypeBuildKey(e.TypeFrom, uniqueName));
      }
      else
      {
        this.Context.Policies.Set<IBuildKeyMappingPolicy>(
          new BuildKeyMappingPolicy(new NamedTypeBuildKey(e.TypeTo)),
          new NamedTypeBuildKey(e.TypeFrom, uniqueName));
      }
    }
  }
}

And you are done. The associated test project assures that lifetimes and InjectionMembers are also respected as well as registered instances are reused. As a nice side effect a call to ResolveAll will now return all instances of all mappings including the default mapping which is otherwise omitted by Unity.

Get the source code for the Remember extension  here (project TecX.Unity folder Mapping and the test suite that shows how to use it in TecX.Unity.Test).

It depends Part 1: Contextual Binding

NInject has a very nifty feature called Contextual Binding. It allows users to define in which context to use a specific type mapping. With NInject that is completely baked into the container out-of-the-box. With Unity you need to do some pull-ups to get it.

First you need to capture the neccessary information from Unity’s build pipeline. Which type was originally requested? What is the target for the object that is currently resolved by the pipeline? What dependencies need to be resolved to be able to create the requested object? How to deal with short-circuits in the build pipeline e.g. when an object has a singleton lifetime? What about out-of-band resolves e.g. when you use InjectionFactories where an instance of IUnityContainer is injected?

This is done by a set of custom BuilderStrategies. They are also used to create the hierarchy of requests. If for example you want to create an instance of Foo that has a dependency on an implementation of IBar that needs an ILog which … well, you get it. The hierarchy of those requests must somehow be represented in the structure of the captured information.

What you get

TecX’ implementation mimics that of NInject to a certain degree. Due to the many differences between those two containers it cannot be a 1:1 port though. This is what TecX’ IRequest looks like:

public interface IRequest
{
  IBuilderContext BuilderContext { get; }
  int Depth { get; }
  IRequest ParentRequest { get; }
  IDictionary<string, object> RequestContext { get; }
  Type Service { get; }
  ITarget Target { get; }
  NamedTypeBuildKey BuildKey { get; }
  NamedTypeBuildKey OriginalBuildKey { get; }
  IRequest CreateChild(Type service, IBuilderContext context);
}

It allows access to infrastructure information like the original BuildKey of the request. The BuildKey after Unity performed the type mapping. The Target property tells you into which target the resolved value will be injected (can either be a parameter of a constructor or method or a property). The parent request, if the current request was made to resolve a sub-dependency. How deep down in the resolve hierarchy we are etc.

The implementation class Request provides access to the current request via the static Current property.

More context

You can register additional context information via the (also static) property StaticRequestContext which is a simple key/value store of type IDictionary<string, object>. This property is important for another feature that uses the contextual binding and will be presented in a later post.

As you can see in the code snippet above IRequest also has a RequestContext property. This can be used to store ‘per request’ information that will be shared throughout a single request but is lost at the end of that request. I decided to go for some more convenience and provide a custom implementation of IDictionary<TKey, TValue> that lets you access values from both the static and the per request context in a read through manner. Per request information has precedence over static information. If no per request information with that key is found the lookup is done on the static context next. If you add some values to the dictionary using IRequest.RequestContext this information will only be added to the per request context.

How to use it

There are several extension methods for IUnityContainer. Below you can see the signature of the method taking the most parameters:

public static IUnityContainer RegisterType(this IUnityContainer container,
  Type @from, Type to, LifetimeManager lifetime,
  Predicate predicate, params InjectionMember[] injectionMembers)

It allows you to specify source and target types for the mapping, the lifetime for the created object, a predicate that is validated to find out when to apply the mapping and a set of optional InjectionMembers where you can specify things like constructor to use, interceptors and much more. Overloads of that method add generic registrations, default lifetime etc.

[Teaser] The contextual binding features are also incorporated into TecX’ enhanced fluent configuration API.

This API lets you do the following:

var builder = new ConfigurationBuilder();
builder.For().Use().When(request => { /* ... */ };

or this:

builder.For<IFoo>().Use<Foo>().WhenInjectedInto<Parent>();

[/Teaser]

TecX’ Contextual Binding gives you more fine grained control over your type mappings than simple named mappings do. It also allows you to build far more readable registrations than with nested InjectionConstructors and ResolvedParameters.

Get the source code for Contextual Binding here (project TecX.Unity folder ContextualBinding and the test suite that shows how to use it in TecX.Unity.ContextualBinding.Test).