Quartz.NET meets Design Patterns

This is the third in a series of posts.

In the last post I showed you how I set up some tests for my implementation of retries with Quartz.NET. I repeatedly hinted at some neat tricks to make things more convenient so here they are.

Quartz.NET requires that your jobs only throw JobExecutionExceptions (as explained at the very bottom of this page). There are reasons why this restriction makes sense but I don’t want to litter my business logic with repetitions of the exact same exception handling code. I think that’s what DRY is all about. I don’t want to force all of my jobs to inherit from a specific base class either. At least not for the single purpose of following Quartz.NET’s rules for exception handling.

But by applying a decorator to my job classes I can fix that once and for all.

public class EnsureJobExecutionExceptionDecorator : IJob
{
  private readonly IJob inner;
  public EnsureJobExecutionExceptionDecorator(IJob inner)
  {
    this.inner = inner;
  }
  public void Execute(IJobExecutionContext context)
  {
    try
    {
      this.inner.Execute(context);
    }
    catch (JobExecutionException)
    {
      throw;
    }
    catch (Exception cause)
    {
      throw new JobExecutionException(cause);
    }
  }
}

JobExecutionExceptions are simply rethrown. Which allows you to throw them in your job if you have to tweak what to tell the scheduler. All other exceptions become InnerExceptions of a new JobExecutionException. Done. Now that was easy.

But how do I ensure that each time Quartz.NET instantiates a job the decorator is in place?

By replacing the scheduler’s default IJobFactory with something more advanced. For my playground I derived from the PropertySettingJobFactory base class and use Unity to create my jobs.

private sealed class UnityJobFactory : PropertySettingJobFactory
{
  private readonly IUnityContainer container;
  public UnityJobFactory(IUnityContainer container)
  {
    this.ThrowIfPropertyNotFound = false;
    this.WarnIfPropertyNotFound = true;
    this.container = container;
  }
  public override IJob NewJob(
    TriggerFiredBundle bundle,
    IScheduler scheduler)
  {
    Type jobType = bundle.JobDetail.JobType;
    IJob job = (IJob)this.container.Resolve(jobType);
    JobDataMap data = new JobDataMap();
    data.PutAll(scheduler.Context);
    data.PutAll(bundle.JobDetail.JobDataMap);
    data.PutAll(bundle.Trigger.JobDataMap);
    this.SetObjectProperties((object)job, data);
    return job;
  }
  public override void ReturnJob(IJob job)
  {
    this.container.Teardown(job);
  }
}

And then its a simple matter to configure Unity to wrap every job it creates with the EnsureJobExecutionExceptionDecorator. Not that hard is it?

And finally there is the code snippet that unfreezes my test thread when I’m done.

public class UnfreezeWhenJobShouldNotRunAgain : IRetryStrategy
{
  private readonly IRetryStrategy inner;
  private readonly ManualResetEvent reset;
  public UnfreezeWhenJobShouldNotRunAgain(
    IRetryStrategy inner,
    ManualResetEvent reset)
  {
    this.inner = inner;
    this.reset = reset;
  }
  public bool ShouldRetry(IJobExecutionContext context)
  {
    bool shouldRetry = this.inner.ShouldRetry(context);
    if (!shouldRetry)
    {
      this.reset.Set();
    }
    return shouldRetry;
  }
  public ITrigger GetTrigger(IJobExecutionContext context)
  {
    return this.inner.GetTrigger(context);
  }
}

Yet another decorator. Whenever the RetryJobListener from the first post of this series queries the IRetryStrategy wether a job should be run again the decorator checks for “yes” or “no”. And in case of a “no” it will set the ManualResetEvent and allow the test thread to continue.

So we have decorators, an abstract factory and dependency injection here. And all of that in less than 200 lines of code. All pieces short and to the point but by combining them you can build mighty powerful solutions that are still clean and easy to understand.

I hope you enjoyed the series and come back for another read. See you soon!

Pipes and Filters

Pipes and filters (or just pipeline) is another common pattern. Oren Eini and Jeremy Likness both have very interesting posts about it on their respective blogs.

While Jeremy’s post aims at the slick creation of pipelines, Oren talks more about the pattern itself and how to implement it in a specific manner (using input and output values of Type IEnumerable<T>).

Some interesting argument came up in the comments to Oren’s post. “Why not use LINQ instead of your custom pipeline (framework)?”

I won’t repeat all the pros and cons here. Better read the posts yourself, they are worth your time!

My opinion on the topic: LINQ is a great tool. But I believe it’s neither the only solution for chains of queries and transformations, nor is it the best in all cases.

I will pick up one point from the “pro LINQ” point of view. “With LINQ you can have different input and output types.”

Sure you can. But who says you can’t do that with pipes and filters just as easily?

We define an abstract base class Filter<TIn, TOut>

public abstract class Filter<TIn, TOut>
{
  public abstract IEnumerable<TOut> Process(IEnumerable<TIn> input);
  public Filter<TIn, TNext> Pipe<TNext>(Filter<TOut, TNext> next)
  {
    return new Pipe<TIn, TOut, TNext>(this, next);
  }
}

and a derived class Pipe<TIn, T, TOut>

public class Pipe<TIn, T, TOut> : Filter<TIn, TOut>
{
  private readonly Filter<TIn, T> source;
  private readonly Filter<T, TOut> destination;
  public Pipe(Filter<TIn, T> source, Filter<T, TOut> destination)
  {
    this.source = source;
    this.destination = destination;
  }
  public override sealed IEnumerable<TOut> Process(IEnumerable<TIn> input)
  {
    var x = this.source.Process(input);
    var result = this.destination.Process(x);
    return result;
  }
}

With these two as a base we can easily chain filters with different input and output types.

We can also fine-tune the ends of the pipeline a bit so that the code is a nicer read. With a small extension method and a just as small dummy class we can use any enumerable as the starting point for defining a pipeline.

public static Filter<TIn, TOut> Pipe<TIn, TOut>(this IEnumerable<TIn> enumerable, Filter<TIn, TOut> filter)
{
  return new PipelineStartDummy<TIn, TOut>(enumerable, filter);
}
private class PipelineStartDummy<TIn, TOut> : Filter<TIn, TOut>
{
  private readonly IEnumerable<TIn> enumerable;
  private readonly Filter<TIn, TOut> filter;
  public PipelineStartDummy(IEnumerable<TIn> enumerable, Filter<TIn, TOut> filter)
  {
    this.enumerable = enumerable;
    this.filter = filter;
  }
  public override IEnumerable<TOut> Process(IEnumerable<TIn> input)
  {
    return this.filter.Process(this.enumerable);
  }
}

If we don’t care what comes out of the pipeline and just want to start processing values from the source we can use another extension method that encapsulates Oren’s enumerator magic.

public static void Start<TIn, TOut>(this Filter<TIn, TOut> filter)
{
  var enumerable = filter.Process(null);
  var enumerator = enumerable.GetEnumerator();
  while (enumerator.MoveNext()) { }
}

And now we put it all together and get the following:

new Numbers(3).Pipe(new Square()).Pipe(new Printer()).Start();

Numbers just returns integer values between 1 and the constructor parameter. Square squares all input values and the Printer writes the input to the console. The call to Start() starts the processing.

While it is not the most impressive example implementation of the pipes and filters pattern I believe that it demonstrates how powerful and flexible the pattern can be. And the code is still readable and very explicit about what you are doing. With just a few lines of code you have a composable, easy to understand solution where you can recombine filters in different orders to change the behavior of the pipeline. And you can do just about anything inside of those filters (think validation or enriching the values traversing through the pipeline with data from services or persistent storage). You can even change the type of the values you are processing between steps. This is yet another case of “Like it a lot!”

Specification Pattern

The Specification Pattern. Yet another one I find useful in daily business. But a few enhancements can make it even more useful and a lot nicer to handle.

1, 2, many!

The CompositeSpecification is most often implemented with two fields for spec1 and spec2 or left and right. The AndSpecification and OrSpecifications evaluate these fields with their respective operator.

public class Or<T> : CompositeSpecification<T>
{
  // ...

  public override bool IsSatisfiedBy(T candidate)
  {
    return spec1.IsSatisfiedBy(candidate) || spec2.IsSatisfiedBy(candidate);
  }
}

This can result in a degenerated tree structure when you link many specifications with the same operator (e.g. a.Or(b).Or(c)...Or(n)).

To avoid this I decided to change the behavior of the composite. Instead of two fields it uses a list of children. If you try to link two specifications the code checks wether either one of them is of the same type as the composite and adds the other composites children instead of the composite itself to the list.

Sounds more complicated than it is. Let’s see some code.

public abstract class CompositeSpecification<T> : Specification<T>
{
  private readonly List<Specification<T>> children;
  public CompositeSpecification()
  {
    this.children = new List<Specification<T>>();
  }
  public IEnumerable<Specification<T>> Children
  {
    get { return children; }
  }
  public int Count
  {
    get { return this.children.Count; }
  }
  protected void Add(Specification<T> specification)
  {
    this.children.Add(specification);
  }
  protected void AddRange(IEnumerable<Specification<T>> specifications)
  {
    this.children.AddRange(specifications);
  }
}

public class Or<T> : CompositeSpecification<T>
{
  public Or(Specification<T> specification, Specification<T> other)
  {
    this.Include(specification);
    this.Include(other);
  }
  public override string Description
  {
    get { return " || "; }
  }
  public override bool IsSatisfiedBy(T candidate)
  {
    foreach (Specification<T> specification in this.Children)
    {
      if (specification.IsSatisfiedBy(candidate))
      {
        return true;
      }
    }
    return false;
  }
  private void Include(Specification<T> specification)
  {
    Or<T> or = specification as Or<T>;
    if (or != null)
    {
      this.AddRange(or.Children);
    }
    else
    {
      this.Add(specification);
    }
  }
}

And this is how two specifications can be linked with an Or operator.

public abstract class Specification<T>
{
  // ...
  
  public abstract bool IsSatisfiedBy(T candidate);
  public Specification<T> Or(Specification<T> other)
  {
    return new Or<T>(this, other);
  }
}

In the end this will put all successive Or’s in a single list, which makes finding the right specification in the tree a lot easier.

What do we have?

Generating a human readable representation of the specification tree can be tedious but is often beneficial if you need to see “what you have”. The easiest way to traverse a tree structure is a Visitor. The same pattern is used by Microsoft in their expression trees.

We add an Accept method and a property for the description of the specification to the base classes

public abstract class Specification<T>
{
  // ...
  public abstract string Description { get; }
  public virtual void Accept(SpecificationVisitor<T> visitor)
  {
    visitor.Visit(this);
  }
}

public abstract class CompositeSpecification<T> : Specification<T>
{
  // ...
  public override void Accept(SpecificationVisitor<T> visitor)
  {
    visitor.Visit(this);
  }
}

public class Or<T> : CompositeSpecification<T>
{
  // ...
  public override string Description
  {
    get { return " || "; }
  }
}

Define a base class for the SpecificationVisitor

public abstract class SpecificationVisitor<T>
{
  public abstract void Visit(Specification<T> specification);
  public abstract void Visit(CompositeSpecification<T> composite);
}

And the implementation of a PrettyPrinter becomes as simple as that

public class PrettyPrinter<T> : SpecificationVisitor<T>
{
  private readonly StringBuilder sb;
  public PrettyPrinter()
  {
    this.sb = new StringBuilder(250);
  }
  public override void Visit(Specification<T> specification)
  {
    this.sb.Append(specification.Description);
  }
  public override void Visit(CompositeSpecification<T> composite)
  {
    this.sb.Append("(");
    foreach (Specification<T> child in composite.Children)
    {
      child.Accept(this);
      this.sb.Append(composite.Description);
    }
    int l = composite.Description.Length;
    this.sb.Remove(sb.Length - l, l);
    this.sb.Append(")");
  }
  public override string ToString()
  {
    return this.sb.ToString();
  }
}

And this gives you a nice and friendly printout of your graph

var spec = new AlwaysFalse().Or(new AlwaysTrue().And(new Odd())).Or(new AlwaysTrue());
var printer = new PrettyPrinter<T>();
spec.Accept(printer);
string friendly = printer.ToString(); // (false || (true && Odd) || true)

First time I tried to attach some zipped source code and found out that WordPress won’t let me… Get the source code here (project TecX.Common folder Specifications and the test suite that puts them to use in TecX.Common.Test).

Rebuttal: Composite Pattern

Oren Eini (aka Ayende Rahien) has an interesting series on his blog where he reviews the GoF Design Patterns and how they apply (or don’t) to modern software development.

As part of this series he also reviews the Composite Pattern and finishes with the following advice:

Recommendation: If your entire dataset can easily fit in memory, and it make sense, go ahead. Most of the time, you probably should stay away.

I find his conclusion somewhat limited as he only seems to take data storage/structures into account. But what about code that “does” things?

If you consider the following interface:

public interface IWriter
{
  void Write(Foo foo);
}

there may be a lot of implementations that write somewhere. Like the local file system. A database. A WCF service or any number of other targets. And what if you want to write to several of those targets and not only one? Do you want to change your code to handle a List<IWriter>? Everywhere? No? Me neither. Instead I prefer to leave my existing code that knows how to write to a single target as is and introduce a composite writer that does the job of writing to multiple targets for me.

public class CompositeWriter : IWriter
{
  private readonly List<IWriter> writers = new List<IWriter>();
  public void Write(Foo foo)
  {
    foreach(var writer in this.writers)
    {
      writer.Write(foo);
    }
  }
  public void Add(IWriter writer)
  {
    this.writers.Add(writer);
  }
}

The sample ignores validation or error handling (Does the composite ignore errors in its targets and just continues? Do the targets have to handle the errors themselves etc.). It also ignores the possible complexity of the Foo it has to write. This is where I agree with Oren. If this structure is large and you need to duplicate it for every Write() you are screwed.

I also ignore the delay that might be introduced by writing to an arbitrarily large set of targets. But you can also handle that inside the composite. Maybe you introduce timeouts for calling a single writer. Or you write a decorator that calls a writer on a background thread if that fits your needs. You can make your solution as complex or simple as you have to. The fact is: Your original code won’t know about that complexity. It just calls an implementation of your IWriter interface. I think this is what SRP is about.

You isolate the callers from the possibly complex task of calling out to multiple targets. Your composite is responsible for dealing with that complexity. And that is it’s only responsibility.

The Builder Pattern

A couple of years ago Jan van Ryjswyck published some articles on the Builder Pattern.

In his series he describes the pattern in the context of test data creation

• Test Data Builder vs. Object Mother
• Test Data Builder
• Test Data Builders Refined

While it is absolutely true that this pattern is very useful when you have to repeatedly create test data with small variations it does not stop there!

Frameworks like Enterprise Library or AutoFac use the same pattern to create their configuration settings.

IConfigurationSource configSource = new DictionaryConfigurationSource();
ConfigurationSourceBuilder builder = new ConfigurationSourceBuilder();

builder.ConfigureLogging()
  .LogToCategoryNamed("DataAccess")
  .SendTo.RollingFile("MyListener")
    .FormatWith(
      new FormatterBuilder()
        .TextFormatterNamed("MyFormatter")
        .UsingTemplate("{timestamp} {message}"));

builder.UpdateConfigurationWithReplace(configSource);
IUnityContainer container = new UnityContainer();
IContainerConfigurator configurator = new UnityContainerConfigurator(container);
EnterpriseLibraryContainer.ConfigureContainer(configurator, configSource);

But it still doesn’t stop there! If you are developing a component that others will have to use, builders are a way to use Dependency Injection in your code but neither enforce the usage of a specific DI container nor the usage of the DI Pattern at all on these consumers.

Mark Seemann, author of Dependency Injection in .NET, outlines how this can be done in his answer to a question on StackOverflow.

You can encapsulate meaningful default values in your builders whitout poluting your API with various constructor overloads. This allows novice consumers to use your components easily while experts can swap the defaults for custom implementations if neccessary or even use a DI container to wire them up and bypass the builders.