Enumeration Classes, WCF and Service Metadata

In a previous post I described the steps to hide enumeration classes from the consumers of a WCF service. That works fine if you manage your client’s contracts manually i.e. each client has its own service contract as well as client versions of all data contracts used in that service. This guarantees the cleanest separation between client and server. But it is not the most convenient way to consume a WCF service. Most projects I have seen use “Add Service Reference” instead to generate proxies from WSDL exposed by a ServiceMetadataEndpoint sometimes refered to as MEX.

And this is where the described steps fail. The metadata export will stil talk about the enumeration class. It will try to serialize the backing fields value, name and displayName. That’s not what we want.

The neccessary information to fix this misbehavior is hidden at the very bottom of Microsoft’s help site about Data Contract Surrogates. The paragraph To Use a Surrogate for Metadata Export describes how you can hook up your custom surrogate with the infrastructure that is used to generate the WSDL for your service. If you include this code snippet in the behavior that decorates your service implementation your clients will only see the generated enum class.

public class HideEnumerationClassesBehavior : Attribute, IServiceBehavior
{
  private readonly IDataContractSurrogate surrogate;
  public HideEnumerationClassesBehavior()
  {
    this.surrogate = new EnumerationClassesSurrogate();
  }
  public void ApplyDispatchBehavior(ServiceDescription serviceDescription, ServiceHostBase host)
  {
    foreach (ServiceEndpoint endpoint in serviceDescription.Endpoints)
    {
      foreach (OperationDescription operation in endpoint.Contract.Operations)
      {
        DataContractSerializerOperationBehavior behavior =
          operation.Behaviors.Find<DataContractSerializerOperationBehavior>();
        if (behavior == null)
        {
          behavior = new DataContractSerializerOperationBehavior(operation)
            { DataContractSurrogate = this.surrogate };
          operation.Behaviors.Add(behavior);
        }
        else
        {
          behavior.DataContractSurrogate = this.surrogate;
        }
      }
    }
    this.HideEnumerationClassesFromMetadata(host);
  }

  private void HideEnumerationClassesFromMetadata(ServiceHostBase host)
  {
    ServiceMetadataBehavior smb = host.Description.Behaviors.Find<ServiceMetadataBehavior>();
    if (smb == null) { return; }
    WsdlExporter exporter = smb.MetadataExporter as WsdlExporter;
    if (exporter == null) { return; }
    object dataContractExporter;
    XsdDataContractExporter xsdInventoryExporter;
    if (!exporter.State.TryGetValue(typeof(XsdDataContractExporter), out dataContractExporter))
    {
      xsdInventoryExporter = new XsdDataContractExporter(exporter.GeneratedXmlSchemas);
    }
    else
    {
      xsdInventoryExporter = (XsdDataContractExporter)dataContractExporter;
    }
    exporter.State.Add(typeof(XsdDataContractExporter), xsdInventoryExporter);
    if (xsdInventoryExporter.Options == null)
    {
      xsdInventoryExporter.Options = new ExportOptions();
    }
    xsdInventoryExporter.Options.DataContractSurrogate = this.surrogate;
  }
  /* not implemented methods omitted for brevity */
}

We are getting there. One blemish that I almost missed is that the enum Type parameters are exported as nullable. I guess this origins from the fact that the enumeration classes are reference and not value types. I needed to find a way around that issue as well.

This article by Carlos Figueira helped a lot.

I added an implementation of IWsdlExportExtension to the above behavior as well as an empty implementation of IContractBehavior. The later is neccessary to hook up the extension with the WCF infrastructure. After a lot of digging around in the contents of the System.Web.Services.Description namespace I found the right hooks to replace method parameters of an enumeration class type with the corresponding enum type. The implementation of the IWsdlExportExtension.ExportContract method now looks like this:

void IWsdlExportExtension.ExportContract(WsdlExporter exporter, WsdlContractConversionContext context)
{
  foreach (var operation in context.Contract.Operations)
  {
    foreach (var message in operation.Messages)
    {
      foreach (var part in message.Body.Parts)
      {
        Type enumType;
        if (this.surrogate.Generator.TryGetEnumByType(part.Type, out enumType))
        {
          part.Type = enumType;
        }
      }
    }
  }
}

And finally we are there. We have completely hidden our server-side enumeration class from the service’s consumers. No matter if we use hand-crafted contracts or proxies generated from service metadata.

The complete source code is available here (project TecX.EnumClasses and the test suite that puts them to use can be found in TecX.EnumClasses.Test). There is a separate solution file TecX.EnumClasses as well. This solution contains additional projects that demonstrate client and server side use.

Enumeration Classes as Flags

In my last post I talked about serializing Enumeration Classes for use with WCF. In this post I’m gonna show you how to add a feature that real enums have but enumeration classes don’t. Yet!

Enums can be decorated with the FlagsAttribute. This allows you to combine single enum values using the ‘|’ operator. A sample from the System.Reflection namespace are BindingFlags.

BindingFlags flags = BindingFlags.Public | BindingFlags.Instance;

To find out wether a single value is part of such a combination you use the ‘&’ operator.

var flags = (BindingFlags.Public | BindingFlags.Instance) & BindingFlags.Public;

The result will be BindingFlags.Public.

To implement the same functionality for enumeration classes we need to define a class that implements both operators ‘|’ and ‘&’. This will be the base class for all flags classes

public abstract class Flags<T> : Enumeration where T : Flags<T>
{
  public Flags(int value, string displayName, string name)
    : base(value, displayName, name)
  {
  }
  public static T operator |(Flags<T> x, Flags<T> y)
  {
    return x.Or(y.ToEnumeration());
  }
  public static T operator &(Flags<T> x, Flags<T> y)
  {
    var f1 = x.ToArray();
    var f2 = y.ToArray();
    var f3 = f1.Intersect(f2).ToArray();
    if (f3.Length == 0)
    {
      return FromValue<T>(0);
    }
    if (f3.Length == 1)
    {
      return f3[0];
    }
    var or = f3[0].Or(f3[1]);
    for (int i = 2; i < f3.Length; i++)
    {
      or = or.Or(f3[i]);
    }
    return or;
  }
  protected internal virtual T[] ToArray()
  {
    return new[] { this.ToEnumeration() };
  }
  protected abstract T ToEnumeration();
  protected abstract T Or(T x);
}

To make our lives easier when implementing flags based on enumeration classes we will also use a generic helper class Or<T>.

public class Or<T> where T : Flags<T>
{
  private readonly List<T> flags;
  public Or(T x, T y)
  {
    this.flags = new List<T>();
    this.flags.AddRange(x.ToArray());
    this.flags.AddRange(y.ToArray());
  }
  public string Name
  {
    get
    {
      string[] names = this.flags.OrderBy(f => f.Value).Select(f => f.Name).ToArray();
      return String.Join(" ", names);
    }
  }
  public int Value
  {
    get { return this.flags.Sum(f => f.Value); }
  }
  public string DisplayName
  {
    get
    {
      string[] names = this.flags.OrderBy(f => f.Value).Select(f => f.DisplayName).ToArray();
      return String.Join(" ", names);
    }
  }
  public T[] ToArray()
  {
    return this.flags.ToArray();
  }
  public void Add(T x)
  {
    this.flags.AddRange(x.ToArray());
  }
}

Now we are armed to build a custom flags class with minimal effort.

public class Numbers : Flags<Numbers>
{
  public static Numbers None = new Numbers(0, "None");
  public static Numbers One = new Numbers(1, "One");
  public static Numbers Two = new Numbers(2, "Two");
  public static Numbers Four = new Numbers(4, "Four");

  private Numbers(int value, string name)
  : base(value, name, name)
  {
  }
  protected override Numbers ToEnumeration()
  {
    return this;
  }
  protected override Numbers Or(Numbers x)
  {
    return new OredNumbers(this, x);
  }
  private class OredNumbers : Numbers
  {
    private readonly Or<Numbers> flags;
    public OredNumbers(Numbers x, Numbers y)
      : base(-1, String.Empty)
    {
      this.flags = new Or<Numbers>(x, y);
    }
    public override string Name
    {
      get { return this.flags.Name; }
    }
    public override string DisplayName
    {
      get { return this.flags.DisplayName; }
    }
    public override int Value
    {
      get { return this.flags.Value; }
    }
    protected internal override Numbers[] ToArray()
    {
      return this.flags.ToArray();
    }
    protected override Numbers Or(Numbers x)
    {
      this.flags.Add(x);
      return this;
    }
  }
}

Admittedly this is much more code than you would have to write for simple flags based on an enum. But the additional power of enumeration classes can also be found in the flags classes. Think of a user selecting from a set of predefined filter criteria. As long as Microsoft does not make expressions serializable you have no way of dynamically building the desired filter on the client side and send it to the server for evaluation other than directly accessing your database (be it SQL or NoSQL) and let some LINQ provider do the translation from expression to native query for you. With flags classes you can combine an abstraction of your filters on the client side, send the flags and let WCF do the reconstruction of the filters on the server side.

The complete source code is available here (project TecX.EnumClasses and the test suite that puts them to use can be found in TecX.EnumClasses.Test). There is a separate solution file TecX.EnumClasses as well.

Enumeration Classes and WCF

During the last week I was working on a real fun project. It started after I stumbled across an old article by Jimmy Bogard, author of the famous AutoMapper library. In his post he talks about Enumeration Classes. What they are, how they work and what their advantages over simple enums are.

He started by stating that a certain question evolved from a discussion on ALT.NET: “What do you do about enumerations crossing service boundaries?”. His post is inspiring but he did not really give an answer to that exact question.

Although WCF is by no means the only way to implement “services” this unanswered question got me thinking: “What if I want to use functionality-enhanced enumeration classes in a WCF service and simple enums on the client side?”

I know that WCF was designed to “talk dirty” when it comes to conversion between client- and server-side objects. It doesn’t matter if you have classes on one and structs on the other side. They don’t have to have the same (number of) fields. Using the DataMemberAttribute you can name the fields differently or even forward information a certain client does not use in its contracts further down the stream when that client calls another service by implementing IExtensibleDataObject in your objects.

But can it talk enums on the client and enumeration classes on the server? As it turns out: Yes it can!

The SortOrder Enumeration Class

For the following sample I will use a modified version of the SortOrder enumeration class I grabbed from the Tarantino project Jimmy mentions in his article. One of the changes I made to the enumeration base class is to introduce a Name property in addition to the DisplayName. That allows you to handle enumeration classes in a very similar way to real enums. I also added non-generic overloads for the FromValue and FromName methods. This is the modified SortOrder enumeration class.

[Serializable]
public class SortOrder : Enumeration
{
  public static readonly SortOrder Ascending = new AscendingSortOrder();
  public static readonly SortOrder Descending = new DescendingSortOrder();
  private readonly IComparer comparer;
  public SortOrder()
  {
  }
  public SortOrder(int value, string displayName, string name, IComparer comparer)
    : base(value, displayName, name)
  {
    this.comparer = comparer;
  }
  public IComparer Comparer
  {
    get { return comparer; }
  }
  private class AscendingSortOrder : SortOrder
  {
    public AscendingSortOrder()
      : base(0, "Sort ascending", "Ascending", StringComparer.OrdinalIgnoreCase)
    {
    }
  }
  private class DescendingSortOrder : SortOrder
  {
    public DescendingSortOrder()
      : base(1, "Sort descending", "Descending", new DescendingStringComparer())
    {
    }
    private class DescendingStringComparer : IComparer
    {
      public int Compare(string x, string y)
      {
        return -StringComparer.OrdinalIgnoreCase.Compare(x, y);
      }
    }
  }
}

Control (De-)Serialization with IDataContractSurrogate

The task is not quite as easy as I thought at first. The WCF infrastructure gives us the IDataContractSurrogate interface. By implementing this interface you can control how WCF serializes and deserializes specific objects.

Serialization is easy. When WCF calls the GetDataContractType method you return typeof(string) and in your implementation of the GetObjectToSerialize method you return the value of the Name property of your enumeration class and the resulting XML looks exactly as if you were serializing a simple enum. But the real deal is the deserialization. WCF matches the incoming serialized enum to the enumeration class. So far so good. But WCF is not able to convert the serialized input value to the matching enumeration class. If you try to use that enumeration type directly without modifying it on the call to GetDataContractTye WCF will fail with an exception. It can’t do the deserialization with the information it has. But it won’t call GetDeserializedObject for help either. If you return typeof(string) WCF will fail as well telling you it can’t cast a string to your enumeration class type. Even if you implement an implicit or explicit cast operator from string to your enumeration you will get that exception. And it also won’t call GetDeserializedObject. A somewhat strange behavior but there you are.

But I was not willing to give up. If WCF can’t handle primitive values like strings and can’t handle your enumeration class directly but it can handle enums, why not try to treat the incoming value as a custom enum and convert that enum to your enumeration class? I know that sounds stupid. You don’t want to write and maintain code for an enumeration class and an enum on the server side. But you don’t have to. Some prior experiments with emitting IL code taught me that it is very easy to generate such an enum on-the-fly.

Reflection.Emit to the Rescue

So what I did is I created an IL generator that checks wether it already emitted an enum type for a specific enumeration class. If not it dynamically creates an enum type that has the same name as the enumeration class plus a short suffix. It also creates an enum member for every static field in your enumeration class (like SortOrder.Ascending) with the name of the field and the value of the enumeration class, decorates type and members with the DataContract- and EnumMemberAttributes and remembers that type for future use.

For SortOrder the generator creates the following enum:

[DataContract(Name="SortOrder")]
public enum SortOrder_Enum
{
  [EnumMember]
  Ascending = 0,
  [EnumMember]
  Descending = 1
}

See no Evil…

And all of a sudden everything works like a charm. The modified surrogate returns the enum type when asked for the DataContractType and you can handle conversion from enum to enumeration class by using the non-generic methods Enumeration.FromValue(Type, int) or Enumeration.FromName(Type, string) in the call to GetDeserializedObject.

public class EnumerationClassesSurrogate : IDataContractSurrogate
{
  private readonly EnumGenerator generator;
  public EnumerationClassesSurrogate()
  {
    this.generator = new EnumGenerator();
  }
  public Type GetDataContractType(Type type)
  {
    if (typeof(Enumeration).IsAssignableFrom(type))
    {
      Type enumType;
      if (this.generator.TryGetEnumByType(type, out enumType))
      {
        return enumType;
      }
    }
    return type;
  }
  public object GetObjectToSerialize(object obj, Type targetType)
  {
    Enumeration enumeration = obj as Enumeration;
    if (enumeration != null)
    {
      Type enumType;
      if (this.generator.TryGetEnumByType(obj.GetType(), out enumType) &&
          Enum.IsDefined(enumType, enumeration.Name))
      {
        return Enum.Parse(enumType, enumeration.Name);
      }
    }
    return obj;
  }
  public object GetDeserializedObject(object obj, Type targetType)
  {
    Type objType = obj.GetType();
    if (this.generator.IsGeneratedEnum(objType) &&
        typeof(Enumeration).IsAssignableFrom(targetType))
    {
      string name = Enum.GetName(objType, obj);
      return Enumeration.FromName(targetType, name);
    }
    return obj;
  }

  // other methods are not implemented for this sample
}

Serialization works in the same way. You map the enumeration class type to the generated enum type and convert the enumeration value to the matching enum value. All of this is completely transparent for client and server code.

[TestMethod]
public void Should_TransparentlyConvert_BetweenSimpleEnumOnClient_AndComplexEnumClassOnServer()
{
  IEnumerable sms = new[]
    {
      new SerializeMe2 { Text = "1" },
      new SerializeMe2 { Text = "3" },
      new SerializeMe2 { Text = "2" }
    };
  using (ServiceHost host = new ServiceHost(typeof(SortingService), new Uri("http://localhost:12345/svc")))
  {
    host.Open();
    var factory = new ChannelFactory(new BasicHttpBinding());
    var proxy = factory.CreateChannel(new EndpointAddress("http://localhost:12345/svc"));
    var result = proxy.Sort(sms, SortOrderEnum.Ascending).ToList();
    Assert.AreEqual("1", result[0].Text);
    Assert.AreEqual("2", result[1].Text);
    Assert.AreEqual("3", result[2].Text);
  }
}

[ServiceContract]
public interface ISortingService
{
  [OperationContract]
  IEnumerable Sort(IEnumerable itemsToSort, SortOrder sortOrder);
}
[ServiceContract(Name = "ISortingService")]
public interface ISortingServiceClient
{
  [OperationContract]
  IEnumerable Sort(IEnumerable itemsToSort, SortOrderEnum sortOrder);
}
[PutDataContractSurrogateBehavior]
public class SortingService : ISortingService
{
  public IEnumerable Sort(IEnumerable itemsToSort, SortOrder sortOrder)
  {
    return itemsToSort.OrderBy(s => s.Text, sortOrder.Comparer);
  }
}
[DataContract(Name = "SortOrder")]
public enum SortOrderEnum
{
  [EnumMember]
  Ascending,
  [EnumMember]
  Descending
}

What you get

Maybe you don’t consider sorting to be a worldshaking example to demonstrate the advantages of enumeration classes. But I believe that, within the boundaries of a HelloWorld! sample, it shows you quite nicely what this pattern has to offer. Enumeration classes are yet another tool in your toolbox. Use them when they seem to fit your requirements.

The posiblity to use simple enums in your clients while you reserve the much more powerful enumeration classes for the server side helps you to decouple client and server and reduce the complexity of your code. See how simple it is to sort values based on the additional information stored in your enumeration classes. The client does not have to know about that. And the server does not need to know that the client doesn’t know…

The complete source code is available here (project TecX.EnumClasses and the test suite that puts them to use can be found in TecX.EnumClasses.Test). There is a separate solution file TecX.EnumClasses as well.

From Static to Flexible

Back in the “good old days” static classes and singletons where widely used patterns. They could be called from anywhere and you didn’t have to spend a thought on dependency management. Or unit testing. Because most often you couldn’t test code that called methods on those objects. They usually needed config files, databases, web-services, message queues, more files, network shares … just about anything. And none of that plays nice with unit tests. Pity that these days can also be called “earlier this year”…

As a “tame” example of such a static object think of the File class. Now imagine you have a component that uses File.Exists(). How do you test that code? Do you really want to create a file whenever you try to run a test of your component?

Lets look at a sample to show how you can still test that code.

public class MyComponent
{
  public void DoIt(string pathToFile)
  {
    if(File.Exists(pathToFile)) { /* ... */ }
    else { /* ... */ }
  }
}

In order to simulate the situation where the file exists you would have to create said file. That’s not what unit tests are for. They should be almost effortless to write. Wrestling with the file systen in a unit test scenario is definitely not effortless.

A common recommendation is to write a wrapper for the File class which is non-static. That would look like this:

public class FileWrapper
{
  public bool Exists(string path)
  {
    return File.Exists(path);
  }
}

But that’s not the whole story. We need a way to simulate the result of FileWrapper.Exists() or we would just add code but won’t gain anything. To do that we lend ourselves the solution Microsoft used for HttpContext. We introduce a base class with all virtual properties and methods.

public class FileBase
{
  public virtual bool Exists(string path)
  {
    throw new NotImplementedException();
  }
}
public class FileWrapper : FileBase
{
  public override bool Exists(string path)
  {
    return File.Exists(path);
  }
}

Now we change our component to use some instance of FileBase instead of the static File class.

public class MyComponent
{
  private readonly FileBase file;
  public MyComponent(FileBase file)
  {
    this.file = file;
  }
  public void DoIt(string pathToFile)
  {
    if(this.file.Exists(pathToFile)) { /* ... */ }
    else  { /* ... */ }
  }
}

We can use Moq or some hand-crafted test double that implements FileBase to simulate the existance or non-existance of said file very easily.

To make things a bit nastier imagine that MyComponent is used throughout a large legacy codebase. You just changed the signature of it’s constructor. Now you would have to change all calls to that constructor immediately or the system will be broken. That’s not an option.

But if we add a constructor overload that delegates to the constructor we just created and uses the default implementation for FileBase we can leave all calls to the default constructor in place but still have the improved testability of the constructor that follows the Dependency Injection Pattern.

public class MyComponent
{
  private readonly FileBase file;
  public MyComponent() : this(new FileWrapper())
  {
  }
  public MyComponent(FileBase file)
  {
    this.file = file;
  }
  // more code
}

I know that this solution is not pure. The DI Pattern advocates the usage of one constructor only that takes all the dependencies of a class. I prefer that as well. But “legacy enterprise systems” are rarely what I’d call clean and sometimes you have to take some dirty intermediate steps towards clean solutions. With the technique outlined above you can improve such systems one step at a time. You can control how far these improvements are visible by adding or removing constructors that create default implmentations for their dependencies and replacing calls to these default constructors one at a time. Rome wasn’t built in a day…

The Invisible Guard

They are called Guards or Validators. It’s their duty to keep your business logic safe. Invalid input values are rejected by throwing exceptions at the caller to notify him about his wrong-doing. Guards enforce the contracts of your API. If you fail to get past them, it’s your problem, not that of the author of said logic.

But do you really want to see them in the StackTrace of each exception they throw? They are not the reason an exception occured. They just contain the location where the exception originated. But the cause for the exception is the call to a method using the wrong parameter values.

Hide Guards in StackTraces

I first saw that code in a colleague’s project. But it is also documented on StackOverflow.

Create a custom Exception that will be thrown by your Guard class. Override the StackTrace property and remove all lines that are related to that Guard class before you return the trace.

public static class Guard
{
  [DebuggerStepThrough]
  public static void AssertNotNull(object param, string paramName)
  {
    if (param == null)
    {
      throw new GuardNotNullException(paramName);
    }
  }
}
public class GuardNotNullException : ArgumentNullException
{
  public GuardNotNullException(string paramName)
    : base(paramName)
  {
  }

  public override string StackTrace
  {
    get
    {
      var stacktrace = base.StackTrace
        .Split(new[] { Environment.NewLine }, StringSplitOptions.RemoveEmptyEntries)
        .Where(l => !l.StartsWith("   at " + typeof(Guard).FullName));
      return string.Join(Environment.NewLine, stacktrace);
    }
  }
}

Hide Guards in the Debugger

With this little trick your Guards won’t show up in StackTraces. But you don’t want to step through them whenever you debug a method they guard either. After all they are not part of the logic of the method they just enforce the method’s invariants.
You might have noticed the DebuggerStepThroughAttribute on the AssertNotNull method. This attribute instructs the Visual Studio debugger to ignore this method completely.

It’s easy. It’s neat. It helps your developers and administrators to focus on the actual source of the problem instead of distracting them with the noise of your validation code.

String Similarity

“Don’t give me what I asked for! Give me what I want!” – How often did you shout that sentence at your computer during the last week?

Sometimes you misspell a word or a name. Sometimes you don’t know for sure how it is spelled but have a vague idea.

Google gives you suggestions when it thinks it knows what you really wanted as does RavenDB and many others.

Fuzzy string matching is an interesting topic. There are a million different algorithms that work better or worse in various scenarios. Just to get a feeling for what is available out there I decided to implement some of the most well-known. Namely these are

• SoundEx (the algorithm described on Wikipedia is the slightly modified American SoundEx)
• Hamming distance
• Levenshtein distance
• Jaccard index
• Damerau-Levenshtein distance
• Dice’s coefficient

The output of these algorithms varies from 4-character strings to values in the range of [0..1] to the number of differences between the compared strings. If you want to make the different algorithms interchangeable in your application you would need to convert these heterogeneous values to a specific numerical range. I made some attempts to that as well but did not put too much effort into it.

It was an interesting experiment!

Get the source code for the above algorithms  here (project TecX.StringSimilarity and the test suite that puts them to use in TecX.StringSimilarity.Test).