Testing custom StyleCop rules

StyleCop is the preferred way to enforce consistency in your project’s coding style. Code guidelines in printed or electronic form are always inferior to automatic validation of said guidelines as part of your build process. You get compiler warnings or errors with (hopefully) helpful and descriptive explanations of why what you did is a bad idea. StyleCop even contains a ReSharper plugin that adds quick-fixes for many rule violations.

But StyleCop can do more than just notify you that someone forgot an empty line after a closing curly bracket. As a proof of concept I wrote rules that should prevent faulty re-throws (catch-blocks in the form of catch(Exception ex) { throw ex; }) that swallow the StackTrace or constructors with more than 4 parameters (which is considered a code smell that indicates a component has too many responsibilities).

The point of this post however is not to show you how to write such rules but how you can make sure that they really work by testing them as part of your regular unit testing process. If you are looking for samples on how to write custom rules check out this post or use the search engine of your choice. There are lots of samples.

Posts like this one explain how to setup tests for custom StyleCop rules using the Visual Studio test-runner and VS’ Test View. In my opinion that thing gives you the worst possible user-experience. It makes running your tests as convenient as reading a paper about quantum physics. ReSharper or TestDriven.Net are way more easy to handle. But the given examples don’t work properly with those tools or alternative test-runners (e.g. NUnit or xUnit). I love my tools and I hate to be forced to use something inferior by circumstances that I can influence. So I set out to remove that particular bump in the road.

I did not want to fumble with files and how and where to deploy them for test-runs with different test frameworks or tools. Instead I wanted to make these files embedded resources that are just there when I run my tests. I started with the sample mentioned above and made several changes to it.

1. Use the StyleCopObjectConsole instead of the StyleCopConsole.
2. Create a class derived from SourceCode that handles loading code from embedded resources (I called mine ResourceBasedSourceCode. Samples for that can be found in the StyleCop source code. Search for StringBasedSourceCode).
3. The StyleCopObjectConsole requires an instance of ObjectBasedEnvironment which in turn requires a delegate that creates SourceCode instances for given file names. Make that delegate create instances of your ResourceBasedSourceCode.
4. Hook up the handlers for output and violations to the events of StyleCopObjectConsole.Core instead of the events of the console object.

Lets have a look at the source code of the tests for which I use xUnit.

public class CodeQualityAnalyzerFixture
{
  private readonly StyleCopObjectConsole console;
  private readonly ICollection output;
  private readonly ICollection violations;
  private readonly CodeProject codeProject;

  public CodeQualityAnalyzerFixture()
  {
    this.violations = new List<string>();
    this.output = new List<string>();
    ObjectBasedEnvironment environment = new ObjectBasedEnvironment(ResourceBasedSourceCode.Create, GetProjectSettings);
    ICollection<string> addinPaths = new[] { "." };
    this.console = new StyleCopObjectConsole(environment, null, addinPaths, true);
    CsParser parser = new CsParser();
    parser.FileTypes.Add("CS");
    this.console.Core.Environment.AddParser(parser);
    this.console.Core.ViolationEncountered += (s, e) =&gt; this.violations.Add(e.Violation);
    this.console.Core.OutputGenerated += (s, e) =&gt; this.output.Add(e.Output);
    this.codeProject = new CodeProject(0, null, new Configuration(null));
  }

  [Fact]
  public void Should_Flag_TooManyCtorParameters()
  {
    this.AnalyzeCodeWithAssertion("TooManyConstructorArguments.cs", 1);
  }

  [Fact]
  public void Should_Flag_IncorrectRethrow()
  {
    this.AnalyzeCodeWithAssertion("IncorrectRethrow.cs", 1);
  }

  private Settings GetProjectSettings(string path, bool readOnly)
  {
    return null;
  }

  private void AnalyzeCodeWithAssertion(string codeFileName, int expectedViolations)
  {
    this.AddSourceCode(codeFileName);
    this.StartAnalysis();
    this.WriteViolationsToConsole();
    this.WriteOutputToConsole();
    Assert.Equal(expectedViolations, this.violations.Count);
  }

  private void WriteOutputToConsole()
  {
    foreach (var o in this.output)
    {
      Console.WriteLine(o);
    }
  }

  private void WriteViolationsToConsole()
  {
    foreach (var violation in this.violations)
    {
      Console.WriteLine(violation.Message);
    }
  }

  private void AddSourceCode(string resourceFileName)
  {
    bool sourceCodeSuccessfullyLoaded = this.console.Core.Environment.AddSourceCode(this.codeProject, resourceFileName, null);
    if (sourceCodeSuccessfullyLoaded == false)
    {
      throw new InvalidOperationException(string.Format("Source file '{0}' could not be added.", resourceFileName));
    }
  }

  private void StartAnalysis()
  {
    IList projects = new[] { this.codeProject };
    this.console.Start(projects);
  }
}

What does it do? Lets start with the test setup in the constructor.

1. Initialize the collections for the output generated by StyleCop.
2. Instantiate the ObjectBasedEnvironment providing it with the required delegates to create SourceCode and Settings. The SourceCodeFactory delegate is a static method of my ResourceBasedSourceCode class so we will come to that in a moment. The ProjectSettingsFactory delegate always returns null.
3. Define a path to the add-in under test (which is the assembly that contains our custom rules). So the dot stands for the “current working folder” where said add-in will be copied to by the build by default.
4. Create the console object and add a CsParser to the console. As the name indicates it is responsible for parsing C# code files. Don’t forget to register the file extension for C# code files when you create the parser or it just won’t work. The extension name is case-sensitive. Use upper-case.
5. Register the eventhandlers that will write output and rule violations to the respective collections that will be evaluated after StyleCop is done processing your code files.
6. Create a new CodePoject. The code files you want analyzed will be added to that project during your tests.

The test methods are simple. They call the AnalyzeCodeWithAssertion method with the name of the file you want to analyze and the number of violations you expect. This is the most basic validation possible. You can make it arbitrarily complex if you want to (e.g. by asserting the type of the expected violation or message texts or …).

Note that I made my ResourceBasedSourceCode class handle incomplete file names. I didn’t want to have to deal with all the namespaces that are added as prefixes to embedded resources so I decided to go with “EndsWith” semantics. As long as the name of the resource ends with the file name provided as a parameter it works just fine.

The AnalyzeCodeWithAssertion method is responsible for

1. Adding the source code to the project that will be analyzed.
2. Running the actual analysis using StyleCop.
3. Writing the output generated during the analysis to the console so you can explore it at leisure after the test is run.
4. Asserting your expectations.

Nothing spectacular there. Now lets have a look at the code for the ResourceBasedSourceCode.

public class ResourceBasedSourceCode : SourceCode
{
  private readonly string _ResourceName;
  public ResourceBasedSourceCode(CodeProject project, SourceParser parser, string resourceName)
    : this(project, parser, resourceName, Assembly.GetExecutingAssembly())
  {
  }
  public ResourceBasedSourceCode(CodeProject project, SourceParser parser, string resourceName, Assembly assembly)
    : base(project, parser)
  {
    Guard.AssertNotEmpty(resourceName, "resourceName");
    Guard.AssertNotNull(assembly, "assembly");
    string fullResourceName = assembly.GetManifestResourceNames()
	  .FirstOrDefault(rn => rn.EndsWith(resourceName, StringComparison.OrdinalIgnoreCase));
	  
    if (string.IsNullOrEmpty(fullResourceName))
    {
      throw new ArgumentException(string.Format("No resource has a name that ends with '{0}'", resourceName), "resourceName");
    }
    _ResourceName = fullResourceName;
  }

  public static SourceCode Create(string path, CodeProject project, SourceParser parser, object context)
  {
    return new ResourceBasedSourceCode(project, parser, path);
  }

  public override TextReader Read()
  {
    Stream stream = this.GetType().Assembly.GetManifestResourceStream(_ResourceName);
    return new StreamReader(stream);
  }

  public override bool Exists
  {
    get { return true; }
  }

  public override string Name
  {
    get { return _ResourceName; }
  }

  public override string Path
  {
    get { return _ResourceName; }
  }

  public override DateTime TimeStamp
  {
    get { return DateTime.MinValue; }
  }

  public override string Type
  {
    get { return "cs"; }
  }
}

There is very little magic involved in this class. In the constructor we try to find an embedded resource matching the name that is provided as a parameter and store the full name of the resource in a field. If that step doesn’t fail we can safely assume that we can open a Stream to read that resource later on. The file extension is hard-coded to C# files (property Type). Lower-case for the extension name is OK here. Don’t ask, I have no clue why the developers decided to make that different from the behavior of the CsParser. The Create method is a factory method with the signature of the SourceCodeFactory delegate as required by the ObjectBasedEnvironment. Everything else is implemented in the most basic way to make the analysis pass.

Now just add a folder that contains the code files to your test project (I called it “Resources”). Make sure to set the files’ Build Action to “Embedded Resource”. And you are ready to roll.

That’s all the infrastructure you need. Doesn’t look that complicated, does it? If you are interested in the complete source code you can find that here. The project names are TecX.Build and TecX.Build.Test. Have fun!

Book Discount Kata

Long time no see. About two months without anything interesting (related to dev topics at least) happening.

Recently I had a look at some of the katas at Coding Dojo. Quite interesting stuff. Today I want to present my shot at the Harry Potter book discount kata.

First things first: I used xUnit and especially xUnit’s data theories for TDD’ing my solution. I took a leave out of the kata’s book and used simple integer arrays to represent the shopping basket. I started with the simplest possible implementation. One class (DiscountCalculator) with a single method (Calculate(int[])). But well … that didn’t get me too far. Basically it solved the problem up to “we have two different books, how much is that?” before I decided that I hate the resulting code.

So I leaned back and thought about the problem a bit more. What I needed was a way to find subsets of the books inside the shopping basket that would maximize the discount. Some kind of partitioning algorithm. After a little back and forth I chose to implement that algorithm as a combination of three simple steps:

1. If the basket is empty, you are done and no more partitioning is needed.
2. If you have 8 books left in the basket and you can form two partitions with 4 distinct books each, you should prefer that to a 5/3 partition.
3. Take as many distinct books as possible from the basket.

This is the code for those three steps:

public class EmptyBasket : IBasketPartitioner
{
  public void Partition(PartitioningContext context)
  {
    if (context.Basket.Length == 0)
    {
      context.Finished = true;
    }
  }
}
public class Prefer44To53 : IBasketPartitioner
{
  public void Partition(PartitioningContext context)
  {
    if (context.Basket.Length == 8)
    {
      List<int> basketCopy = new List<int>(context.Basket);
      int[] part1 = context.Basket.Distinct().Take(4).ToArray();
      if (part1.Length == 4)
      {
        foreach (int book in part1)
        {
	      basketCopy.Remove(book);
        }
        int[] part2 = basketCopy.Distinct().ToArray();
        if (part2.Length == 4)
        {
          context.MakePartition(part1);
          context.MakePartition(part2);
        }
      }
    }
  }
}
public class GreedyGrabDistinctBooks : IBasketPartitioner
{
  public void Partition(PartitioningContext context)
  {
    int[] differentBooks = context.Basket.Distinct().ToArray();
    if (differentBooks.Length > 0)
    {
      context.MakePartition(differentBooks);
    }
  }
}

Admittedly the implementation of the 4/4 rule could use some polishing. But it works for now.

To host those steps I used a variation of the chain-of-responsibility pattern. This chain would loop through the different steps. Each step would take some of the books from the basket and put them in a list of partitions until there are no books left. The order of the steps is important! To achieve the desired outcome of the “prefer 4/4 partition to 5/3 partition” rule you need to take those books from the basket before the greedy “take as many distinct books as possible” rule applies. I chose to remove both 4/4 chunks from the basket in step 2 to reduce the overhead of the calls to Distinct().

public class PartitionerChain
{
  private readonly List<IBasketPartitioner> partitioners;
  public PartitionerChain(params IBasketPartitioner[] partitioners)
  {
    this.partitioners = new List<IBasketPartitioner>(partitioners);
  }
  public IEnumerable<int[]> GetPartitions(int[] originalBasket)
  {
    var context = new PartitioningContext(originalBasket);
    int index = 0;
    do
    {
      this.partitioners[index].Partition(context);
      index = (index + 1) % this.partitioners.Count;
    }
    while (!context.Finished && context.Basket.Length > 0);
    return context.Partitions;
  }
}

The chain hands a context from step to step which contains the current content of the shopping basket, a list of partitions and a flag that indicates when the partitioning process is finished.

public class PartitioningContext
{
  private readonly List<int[]> basketPartitions;
  public PartitioningContext(int[] originalBasket)
  {
    this.Basket = originalBasket;
    this.basketPartitions = new List<int[]>();
  }
  public int[] Basket { get; private set; }
  public bool Finished { get; set; }
  public IEnumerable<int[]> Partitions { get { return this.basketPartitions; } }
  public void MakePartition(int[] partition)
  {
    this.basketPartitions.Add(partition);
    List<int> newBasket = new List<int>(this.Basket);
    foreach (int book in partition)
    {
      newBasket.Remove(book);
    }
    this.Basket = newBasket.ToArray();
  }
}

To calculate the actual price of the books I switched from the switch-case solution to (yet again) a chain-of-responsibility based one.

public abstract class DiscountStrategy
{
  public DiscountStrategy Next { get; protected set; }
  public abstract double GetPrice(int[] basket);
}
public class NoDiscount : DiscountStrategy
{
  public override double GetPrice(int[] basket)
  {
    return basket.Sum(book => 8.0);
  }
}
public class TwoBooks : DiscountStrategy
{
  public TwoBooks(DiscountStrategy next)
  {
    this.Next = next;
  }
  public override double GetPrice(int[] basket)
  {
    if (basket.Length == 2)
    {
      return 2 * 8 * 0.95;
    }
    return this.Next.GetPrice(basket);
  }
}
public class ThreeBooks : DiscountStrategy
{
  public ThreeBooks(DiscountStrategy next)
  {
    this.Next = next;
  }
  public override double GetPrice(int[] basket)
  {
    if (basket.Length == 3)
    {
      return 3 * 8 * 0.9;
    }
    return this.Next.GetPrice(basket);
  }
}
public class FourBooks : DiscountStrategy
{
  public FourBooks(DiscountStrategy next)
  {
    this.Next = next;
  }
  public override double GetPrice(int[] basket)
  {
    if (basket.Length == 4)
    {
      return 4 * 8 * 0.8;
    }
    return this.Next.GetPrice(basket);
  }
}    
public class FiveBooks : DiscountStrategy
{
  public FiveBooks(DiscountStrategy next)
  {
    this.Next = next;
  }
  public override double GetPrice(int[] basket)
  {
    if (basket.Length == 5)
    {
      return 5 * 8 * 0.75;
    }
    return this.Next.GetPrice(basket);
  }
}

[OT] Did I mention that I LOVE the chain-of-responsibility pattern? It is super flexible. It allows for clear separation of concerns. Favors small, easy to understand (and test) classes. Changing the behavior of your solution becomes a simple matter of reordering steps that you have already implemented. [/OT]

By this you can easily swap out different discount rules.

After that the calculator was a rather dumb shell. It assembles the two chains in its constructor. This can be seen as a violation of the D(ependency Inversion) of SOLID software development. I chose to encapsulate the knowledge of how to order the different pieces in the chains there nonetheless. If I ever need to make that step configurable, it would be a no-brainer as the assignment already happens in the calculator’s constructor.

All the calculator has to do now is to let the partioners divide the shopping basket into handy pieces and then let the discount strategies calculate the price for the individual chunks. Sweet!

public class DiscountCalculator
{
  private readonly PartitionerChain partitioners;
  private readonly DiscountStrategy discounts;
  public DiscountCalculator()
  {
    this.partitioners = new PartitionerChain(new EmptyBasket(), new Prefer44To53(), new GreedyGrabDistinctBooks());
    this.discounts = new FiveBooks(new FourBooks(new ThreeBooks(new TwoBooks(new NoDiscount()))));
  }
  public double Calculate(int[] basket)
  {
    var partitions = this.partitioners.GetPartitions(basket);
    double total = partitions.Sum(partition => this.discounts.GetPrice(partition));
    return total;
  }
}

One more word to the testing aspect. I mentioned that I used xUnit data theories. With these it was almost effortless to use the test cases described at the bottom of the kata.

[Theory]
[InlineData(0d, new int[0])]
[InlineData(8d, new[] { 0 })]

// ...

[InlineData(2 * 8 * 4 * 0.8, new[] { 0, 0, 1, 1, 2, 2, 3, 4 })]
[InlineData(3 * 8 * 5 * 0.75 + 2 * 8 * 4 * 0.8, new[]
                                                {
                                                  0, 0, 0, 0, 0,
                                                  1, 1, 1, 1,1,
                                                  2, 2, 2, 2,
                                                  3, 3, 3, 3, 3,
                                                  4, 4, 4, 4
})]
public void CalculatesCorrectPrice(double expected, int[] basket)
{
  DiscountCalculator sut = new DiscountCalculator();
  Assert.Equal(expected, sut.Calculate(basket));
}

It was never ever that easy to setup tests that use different data but are equivalent otherwise. If you are interested in data theories and how they can make your life as a tester so much easier I strongly recommend that you have a look at Mark Seemann’s awesome series of posts about his implementation of the String Calculator kata. Mind blowing!

So that’s it for now. I think I will have a look at the other katas. Hope they are as much fun 🙂