Using Expressions in WCF

Most business applications deal with showing, analyzing and modifying some kind of business relevant data. Although it is not a rare thing that customers ask you to “load everything” into a grid and let them work the data like they “always did in Excel” this is rarely a good idea. As soon as you have more than “Hello World!” numbers of records in your persistent storage, loading that data will take time, network bandwidth, CPU cycles for (de)serialization and disk I/O etc. pp.

A much better approach is to limit the number of records to be returned as early as possible by filtering out those records that don’t match your criteria. In a client/server application that means doing the filtering on the server. If you use WCF for the communication between client and server there is a slight glitch you need to work around: How do you get the filtering criteria to the server?

Out of the box you can’t transmit the Expression trees that represent your queries over the wire. They are not serializable. Using strings is tedious and prone to typos. In addition you will have to parse your query string unless you use some vendor specific format like Hibernate’s HQL which adds vulnerabilities like “xQL injection” to the mix. Creating server side methods for each filter you can think of is inflexible (i.e. users can’t create dynamic queries) and will result in a lot of code for something that is as easy as a one-line expression would be. Using some implementation of the Specification pattern would also require you to write quite some amount of code for the different search criteria which would only duplicate a subset of the features that Expressions offer. In addition you would add yet another syntax that your co-developers need to master. So unless you don’t mind to write, test, fix, document, update and maintain a lot of code you are back to square one.

Serialize.Linq to the rescue

The problem of Expressions not being serializable is as old as LINQ itself. This article on MSDN dates as far back as 2008. The author uses XML to represent the original query. Another article in a recent edition of a German .NET magazine uses a custom IQueryProvider that performs the translation between Expression tree and XML. Both implementations work to a certain degree but I have to admit I like neither one. My personal opinion on XML: It’s one huge PITA to work with. Sorry for the language. So I try to avoid having to deal with it whenever possible.

Serialize.Linq works a bit different: It maps each node of the original expression tree to a node that the DataContractSerializer can easily digest (I know that means XML somewhere under the covers, but the chances that I will ever see it or have to deal with it are close to zero). In a WCF scenario, this structure is then serialized, transmitted over the wire, deserialized into Serialize.Linq’s custom node format and from there back to a real Expression tree. You can find some articles that describe features of the framework on Sascha Kiefer’s blog.

This article talks about using Serialize.Linq together with WCF. It’s easy and requires almost zero work on your side. You just have to do the conversion between Expressions and SL’s ExpressionNodes whenever you query for data. Over and over again. Hm… Wouldn’t it be a lot better if I could save that effort as well?

I think I got the idea from various posts on Oren Eini’s blog: If you have repetitive tasks that are always the same put them into your infrastructure to relieve the burden on the developers a little. Well, considering that my infrastructure in this case is WCF which has about a bazillion extension points and that I have some experience with messing with WCF’s serialization behavior from my experiments with Enumeration classes I thought I would give it a shot.

Yet another IDataContractSurrogate

I found out that I can reuse most of the code I wrote for the custom serialization of Enumeration classes. By replacing Enumerations with Expressions I got it working in almost no time on the server side. The code snippets ignore methods that are not implemented for brevity.

public class SerializeExpressionsBehavior : Attribute, IServiceBehavior, IContractBehavior, IWsdlExportExtension
{
  private readonly ExpressionDataContractSurrogate surrogate;
  public SerializeExpressionsBehavior()
  {
    this.surrogate = new ExpressionDataContractSurrogate();
  }
  void IServiceBehavior.ApplyDispatchBehavior(ServiceDescription serviceDescription, ServiceHostBase host)
  {
    foreach (ServiceEndpoint endpoint in serviceDescription.Endpoints)
    {
      foreach (OperationDescription operation in endpoint.Contract.Operations)
      {
        DataContractSerializerOperationBehavior behavior = operation.Behaviors.Find();
        if (behavior == null)
        {
          behavior = new DataContractSerializerOperationBehavior(operation) { DataContractSurrogate = this.surrogate };
          operation.Behaviors.Add(behavior);
        }
        else
        {
          behavior.DataContractSurrogate = this.surrogate;
        }
      }
    }
    this.HideEnumerationClassesFromMetadata(host);
  }
  private void HideEnumerationClassesFromMetadata(ServiceHostBase host)
  {
    ServiceMetadataBehavior smb = host.Description.Behaviors.Find();
    if (smb == null)
    {
      return;
    }
    WsdlExporter exporter = smb.MetadataExporter as WsdlExporter;
    if (exporter == null)
    {
      return;
    }
    object dataContractExporter;
    XsdDataContractExporter xsdInventoryExporter;
    if (!exporter.State.TryGetValue(typeof(XsdDataContractExporter), out dataContractExporter))
    {
      xsdInventoryExporter = new XsdDataContractExporter(exporter.GeneratedXmlSchemas);
    }
    else
    {
      xsdInventoryExporter = (XsdDataContractExporter)dataContractExporter;
    }
    exporter.State.Add(typeof(XsdDataContractExporter), xsdInventoryExporter);
    if (xsdInventoryExporter.Options == null)
    {
      xsdInventoryExporter.Options = new ExportOptions();
    }
    xsdInventoryExporter.Options.DataContractSurrogate = this.surrogate;
  }
}

And the DataContractSurrogate that performs the actual translation between Expressions and ExpressionNodes.

public class ExpressionDataContractSurrogate : IDataContractSurrogate
{
  public Type GetDataContractType(Type type)
  {
    if (typeof(Expression).IsAssignableFrom(type))
    {
      return typeof(ExpressionNode);
    }
    return type;
  }
  public object GetObjectToSerialize(object obj, Type targetType)
  {
    Expression expression = obj as Expression;
    if (expression != null)
    {
      return expression.ToExpressionNode();
    }
    return obj;
  }
  public object GetDeserializedObject(object obj, Type targetType)
  {
    if (typeof(Expression).IsAssignableFrom(targetType))
    {
      return ((ExpressionNode)obj).ToExpression();
    }
    return obj;
  }
}

As I wanted to use Expressions on the client side as well I needed a new EndpointBehavior to hook up the surrogate. After that … it just worked 🙂

public class ClientSideSerializeExpressionsBehavior : IEndpointBehavior
{
  private IDataContractSurrogate surrogate;
  public ClientSideSerializeExpressionsBehavior()
  {
    this.surrogate = new ExpressionDataContractSurrogate();
  }
  public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
  {
    foreach (OperationDescription operation in endpoint.Contract.Operations)
    {
      DataContractSerializerOperationBehavior behavior = operation.Behaviors.Find();
      if (behavior == null)
      {
        behavior = new DataContractSerializerOperationBehavior(operation) { DataContractSurrogate = this.surrogate };
        operation.Behaviors.Add(behavior);
      }
      else
      {
        behavior.DataContractSurrogate = this.surrogate;
      }
    }
  }
}

So far I tested my solution with proxies derived from ClientBase<T>

using (var proxy = new MyServiceClient(new BasicHttpBinding(), new EndpointAddress(ServiceAddress)))
{
  Customer[] customers = proxy.QueryCustomers(c => c.Id > 3);
  // ...
}

(the client side behavior is added in the proxies constructor)

public MyServiceClient(Binding binding, EndpointAddress address)
  : base(binding, address)
{
  this.Endpoint.Behaviors.Add(new ClientSideSerializeExpressionsBehavior());
}

and those generated on-the-fly by a ChannelFactory<T>. Works like a charm 🙂

var factory = new ChannelFactory(new BasicHttpBinding(), ServiceAddress);
factory.Endpoint.Behaviors.Add(new ClientSideSerializeExpressionsBehavior());
var proxy = factory.CreateChannel();
Customer[] customers = proxy.QueryCustomers(c => c.Id > 3);

What you get and what’s missing

What you get is this beauty:

var customers = proxy.QueryCustomers(c => c.Id > 3);

Neat and simple.

With Serialize.Linq and the small additions to the WCF infrastructure outlined above you can use Expressions when querying a WCF service without breaking your workflow by manually translating queries to another format. Just use the same queries that you would use for any other data source. Done.

I have yet to evaluate wether the above code works with clients generated using “Add service reference”. But that will be the topic of another article. What’s definitely not working is seamlessly using Expressions in Silverlight clients. DataContractSurrogates are among the features that MS decided not to support in Silverlight. *sigh*

As usual you can grab the code from my page on CodePlex (projects TecX.Expressions and TecX.Expressions.Test). Enjoy! 🙂

Advertisement

Enumeration Classes, WCF and Service Metadata

In a previous post I described the steps to hide enumeration classes from the consumers of a WCF service. That works fine if you manage your client’s contracts manually i.e. each client has its own service contract as well as client versions of all data contracts used in that service. This guarantees the cleanest separation between client and server. But it is not the most convenient way to consume a WCF service. Most projects I have seen use “Add Service Reference” instead to generate proxies from WSDL exposed by a ServiceMetadataEndpoint sometimes refered to as MEX.

And this is where the described steps fail. The metadata export will stil talk about the enumeration class. It will try to serialize the backing fields value, name and displayName. That’s not what we want.

The neccessary information to fix this misbehavior is hidden at the very bottom of Microsoft’s help site about Data Contract Surrogates. The paragraph To Use a Surrogate for Metadata Export describes how you can hook up your custom surrogate with the infrastructure that is used to generate the WSDL for your service. If you include this code snippet in the behavior that decorates your service implementation your clients will only see the generated enum class.

public class HideEnumerationClassesBehavior : Attribute, IServiceBehavior
{
  private readonly IDataContractSurrogate surrogate;
  public HideEnumerationClassesBehavior()
  {
    this.surrogate = new EnumerationClassesSurrogate();
  }
  public void ApplyDispatchBehavior(ServiceDescription serviceDescription, ServiceHostBase host)
  {
    foreach (ServiceEndpoint endpoint in serviceDescription.Endpoints)
    {
      foreach (OperationDescription operation in endpoint.Contract.Operations)
      {
        DataContractSerializerOperationBehavior behavior =
          operation.Behaviors.Find<DataContractSerializerOperationBehavior>();
        if (behavior == null)
        {
          behavior = new DataContractSerializerOperationBehavior(operation)
            { DataContractSurrogate = this.surrogate };
          operation.Behaviors.Add(behavior);
        }
        else
        {
          behavior.DataContractSurrogate = this.surrogate;
        }
      }
    }
    this.HideEnumerationClassesFromMetadata(host);
  }

  private void HideEnumerationClassesFromMetadata(ServiceHostBase host)
  {
    ServiceMetadataBehavior smb = host.Description.Behaviors.Find<ServiceMetadataBehavior>();
    if (smb == null) { return; }
    WsdlExporter exporter = smb.MetadataExporter as WsdlExporter;
    if (exporter == null) { return; }
    object dataContractExporter;
    XsdDataContractExporter xsdInventoryExporter;
    if (!exporter.State.TryGetValue(typeof(XsdDataContractExporter), out dataContractExporter))
    {
      xsdInventoryExporter = new XsdDataContractExporter(exporter.GeneratedXmlSchemas);
    }
    else
    {
      xsdInventoryExporter = (XsdDataContractExporter)dataContractExporter;
    }
    exporter.State.Add(typeof(XsdDataContractExporter), xsdInventoryExporter);
    if (xsdInventoryExporter.Options == null)
    {
      xsdInventoryExporter.Options = new ExportOptions();
    }
    xsdInventoryExporter.Options.DataContractSurrogate = this.surrogate;
  }
  /* not implemented methods omitted for brevity */
}

We are getting there. One blemish that I almost missed is that the enum Type parameters are exported as nullable. I guess this origins from the fact that the enumeration classes are reference and not value types. I needed to find a way around that issue as well.

This article by Carlos Figueira helped a lot.

I added an implementation of IWsdlExportExtension to the above behavior as well as an empty implementation of IContractBehavior. The later is neccessary to hook up the extension with the WCF infrastructure. After a lot of digging around in the contents of the System.Web.Services.Description namespace I found the right hooks to replace method parameters of an enumeration class type with the corresponding enum type. The implementation of the IWsdlExportExtension.ExportContract method now looks like this:

void IWsdlExportExtension.ExportContract(WsdlExporter exporter, WsdlContractConversionContext context)
{
  foreach (var operation in context.Contract.Operations)
  {
    foreach (var message in operation.Messages)
    {
      foreach (var part in message.Body.Parts)
      {
        Type enumType;
        if (this.surrogate.Generator.TryGetEnumByType(part.Type, out enumType))
        {
          part.Type = enumType;
        }
      }
    }
  }
}

And finally we are there. We have completely hidden our server-side enumeration class from the service’s consumers. No matter if we use hand-crafted contracts or proxies generated from service metadata.

The complete source code is available here (project TecX.EnumClasses and the test suite that puts them to use can be found in TecX.EnumClasses.Test). There is a separate solution file TecX.EnumClasses as well. This solution contains additional projects that demonstrate client and server side use.

Enumeration Classes and WCF

During the last week I was working on a real fun project. It started after I stumbled across an old article by Jimmy Bogard, author of the famous AutoMapper library. In his post he talks about Enumeration Classes. What they are, how they work and what their advantages over simple enums are.

He started by stating that a certain question evolved from a discussion on ALT.NET: “What do you do about enumerations crossing service boundaries?”. His post is inspiring but he did not really give an answer to that exact question.

Although WCF is by no means the only way to implement “services” this unanswered question got me thinking: “What if I want to use functionality-enhanced enumeration classes in a WCF service and simple enums on the client side?”

I know that WCF was designed to “talk dirty” when it comes to conversion between client- and server-side objects. It doesn’t matter if you have classes on one and structs on the other side. They don’t have to have the same (number of) fields. Using the DataMemberAttribute you can name the fields differently or even forward information a certain client does not use in its contracts further down the stream when that client calls another service by implementing IExtensibleDataObject in your objects.

But can it talk enums on the client and enumeration classes on the server? As it turns out: Yes it can!

The SortOrder Enumeration Class

For the following sample I will use a modified version of the SortOrder enumeration class I grabbed from the Tarantino project Jimmy mentions in his article. One of the changes I made to the enumeration base class is to introduce a Name property in addition to the DisplayName. That allows you to handle enumeration classes in a very similar way to real enums. I also added non-generic overloads for the FromValue and FromName methods. This is the modified SortOrder enumeration class.

[Serializable]
public class SortOrder : Enumeration
{
  public static readonly SortOrder Ascending = new AscendingSortOrder();
  public static readonly SortOrder Descending = new DescendingSortOrder();
  private readonly IComparer comparer;
  public SortOrder()
  {
  }
  public SortOrder(int value, string displayName, string name, IComparer comparer)
    : base(value, displayName, name)
  {
    this.comparer = comparer;
  }
  public IComparer Comparer
  {
    get { return comparer; }
  }
  private class AscendingSortOrder : SortOrder
  {
    public AscendingSortOrder()
      : base(0, "Sort ascending", "Ascending", StringComparer.OrdinalIgnoreCase)
    {
    }
  }
  private class DescendingSortOrder : SortOrder
  {
    public DescendingSortOrder()
      : base(1, "Sort descending", "Descending", new DescendingStringComparer())
    {
    }
    private class DescendingStringComparer : IComparer
    {
      public int Compare(string x, string y)
      {
        return -StringComparer.OrdinalIgnoreCase.Compare(x, y);
      }
    }
  }
}

Control (De-)Serialization with IDataContractSurrogate

The task is not quite as easy as I thought at first. The WCF infrastructure gives us the IDataContractSurrogate interface. By implementing this interface you can control how WCF serializes and deserializes specific objects.

Serialization is easy. When WCF calls the GetDataContractType method you return typeof(string) and in your implementation of the GetObjectToSerialize method you return the value of the Name property of your enumeration class and the resulting XML looks exactly as if you were serializing a simple enum. But the real deal is the deserialization. WCF matches the incoming serialized enum to the enumeration class. So far so good. But WCF is not able to convert the serialized input value to the matching enumeration class. If you try to use that enumeration type directly without modifying it on the call to GetDataContractTye WCF will fail with an exception. It can’t do the deserialization with the information it has. But it won’t call GetDeserializedObject for help either. If you return typeof(string) WCF will fail as well telling you it can’t cast a string to your enumeration class type. Even if you implement an implicit or explicit cast operator from string to your enumeration you will get that exception. And it also won’t call GetDeserializedObject. A somewhat strange behavior but there you are.

But I was not willing to give up. If WCF can’t handle primitive values like strings and can’t handle your enumeration class directly but it can handle enums, why not try to treat the incoming value as a custom enum and convert that enum to your enumeration class? I know that sounds stupid. You don’t want to write and maintain code for an enumeration class and an enum on the server side. But you don’t have to. Some prior experiments with emitting IL code taught me that it is very easy to generate such an enum on-the-fly.

Reflection.Emit to the Rescue

So what I did is I created an IL generator that checks wether it already emitted an enum type for a specific enumeration class. If not it dynamically creates an enum type that has the same name as the enumeration class plus a short suffix. It also creates an enum member for every static field in your enumeration class (like SortOrder.Ascending) with the name of the field and the value of the enumeration class, decorates type and members with the DataContract- and EnumMemberAttributes and remembers that type for future use.

For SortOrder the generator creates the following enum:

[DataContract(Name="SortOrder")]
public enum SortOrder_Enum
{
  [EnumMember]
  Ascending = 0,
  [EnumMember]
  Descending = 1
}

See no Evil…

And all of a sudden everything works like a charm. The modified surrogate returns the enum type when asked for the DataContractType and you can handle conversion from enum to enumeration class by using the non-generic methods Enumeration.FromValue(Type, int) or Enumeration.FromName(Type, string) in the call to GetDeserializedObject.

public class EnumerationClassesSurrogate : IDataContractSurrogate
{
  private readonly EnumGenerator generator;
  public EnumerationClassesSurrogate()
  {
    this.generator = new EnumGenerator();
  }
  public Type GetDataContractType(Type type)
  {
    if (typeof(Enumeration).IsAssignableFrom(type))
    {
      Type enumType;
      if (this.generator.TryGetEnumByType(type, out enumType))
      {
        return enumType;
      }
    }
    return type;
  }
  public object GetObjectToSerialize(object obj, Type targetType)
  {
    Enumeration enumeration = obj as Enumeration;
    if (enumeration != null)
    {
      Type enumType;
      if (this.generator.TryGetEnumByType(obj.GetType(), out enumType) &&
          Enum.IsDefined(enumType, enumeration.Name))
      {
        return Enum.Parse(enumType, enumeration.Name);
      }
    }
    return obj;
  }
  public object GetDeserializedObject(object obj, Type targetType)
  {
    Type objType = obj.GetType();
    if (this.generator.IsGeneratedEnum(objType) &&
        typeof(Enumeration).IsAssignableFrom(targetType))
    {
      string name = Enum.GetName(objType, obj);
      return Enumeration.FromName(targetType, name);
    }
    return obj;
  }

  // other methods are not implemented for this sample
}

Serialization works in the same way. You map the enumeration class type to the generated enum type and convert the enumeration value to the matching enum value. All of this is completely transparent for client and server code.

[TestMethod]
public void Should_TransparentlyConvert_BetweenSimpleEnumOnClient_AndComplexEnumClassOnServer()
{
  IEnumerable sms = new[]
    {
      new SerializeMe2 { Text = "1" },
      new SerializeMe2 { Text = "3" },
      new SerializeMe2 { Text = "2" }
    };
  using (ServiceHost host = new ServiceHost(typeof(SortingService), new Uri("http://localhost:12345/svc")))
  {
    host.Open();
    var factory = new ChannelFactory(new BasicHttpBinding());
    var proxy = factory.CreateChannel(new EndpointAddress("http://localhost:12345/svc"));
    var result = proxy.Sort(sms, SortOrderEnum.Ascending).ToList();
    Assert.AreEqual("1", result[0].Text);
    Assert.AreEqual("2", result[1].Text);
    Assert.AreEqual("3", result[2].Text);
  }
}

[ServiceContract]
public interface ISortingService
{
  [OperationContract]
  IEnumerable Sort(IEnumerable itemsToSort, SortOrder sortOrder);
}
[ServiceContract(Name = "ISortingService")]
public interface ISortingServiceClient
{
  [OperationContract]
  IEnumerable Sort(IEnumerable itemsToSort, SortOrderEnum sortOrder);
}
[PutDataContractSurrogateBehavior]
public class SortingService : ISortingService
{
  public IEnumerable Sort(IEnumerable itemsToSort, SortOrder sortOrder)
  {
    return itemsToSort.OrderBy(s => s.Text, sortOrder.Comparer);
  }
}
[DataContract(Name = "SortOrder")]
public enum SortOrderEnum
{
  [EnumMember]
  Ascending,
  [EnumMember]
  Descending
}

What you get

Maybe you don’t consider sorting to be a worldshaking example to demonstrate the advantages of enumeration classes. But I believe that, within the boundaries of a HelloWorld! sample, it shows you quite nicely what this pattern has to offer. Enumeration classes are yet another tool in your toolbox. Use them when they seem to fit your requirements.

The posiblity to use simple enums in your clients while you reserve the much more powerful enumeration classes for the server side helps you to decouple client and server and reduce the complexity of your code. See how simple it is to sort values based on the additional information stored in your enumeration classes. The client does not have to know about that. And the server does not need to know that the client doesn’t know…

The complete source code is available here (project TecX.EnumClasses and the test suite that puts them to use can be found in TecX.EnumClasses.Test). There is a separate solution file TecX.EnumClasses as well.