Separation of Concerns with Domain Events

Take a look at this nasty code:

[HttpPost, UnitOfWork]
public ActionResult Confirm(Order order)
{
    order.OrderStatus = OrderStatus.Created;

    if(order.ContactMe)
    {
        var mailingListSubscription = new MailingListSubscription
        {
            Contact = order.PostalContact,
            Email = order.Email,
            DateSubscribed = DateTime.Now
        };

        mailingListRepository.SaveOrUpdate(mailingListSubscription);
    }

    EmailOrder(order);
    basketService.CreateNewBasketFor(userService.CurrentUser);
    
    return this.RedirectToAction<OrderController>(c => c.Item(order.Id));
}

This is a controller action from Suteki Shop that is fired when the user confirms an order. There is no separation of concerns (SOC) here. In a single action method we change the status of the order, check to see if the user should be added to our mailing list, email the order confirmation, and finally create a new basket for the user. What we should really be doing here is simply asking the order to update its status:

[HttpPost, UnitOfWork]
public ActionResult Confirm(Order order)
{
    userService.CurrentUser.EnsureCanView(order);
    order.Confirm();

    return this.RedirectToAction<OrderController>(c => c.Item(order.Id));
}

This is much nicer. But the actions that occur as a result of the order confirmation still need to happen. We have a dilemma now, good design says we shouldn’t inject services into our domain entities (Order in this case), but in order to update the mailing list, send an email and create a new basket, we have no choice but to invoke those domain services. Even if we could access domain services from our entities, it would be another SOC violation to do it. Why should the order confirm method have to know about all the things that need to happen when an order is confirmed?

A really nice answer to this kind of issue is Udi Dahan’s Domain Event pattern. This pattern allows domain entities to raise events without having to be aware of what the events will trigger. So the Order.Confirm() method looks like this:

public virtual void Confirm()
{
    OrderStatus = OrderStatus.Created;
    DomainEvent.Raise(new OrderConfirmed(this));
}

It simply alters its state as required, in this case setting its status to Created and then raises an event to tell the world what’s happened.

Events can be handled by zero or more handlers. Handlers implement a simple interface:

public interface IHandle<TEvent> where TEvent : class, IDomainEvent
{
    void Handle(TEvent @event);
}

Here’s the AddToMailingListOnOrderConfirmed handler that adds the user to our mailing list:

public class AddToMailingListOnOrderConfirmed : IHandle<OrderConfirmed>
{
    readonly IRepository<MailingListSubscription> mailingListRepository;

    public AddToMailingListOnOrderConfirmed(IRepository<MailingListSubscription> mailingListRepository)
    {
        this.mailingListRepository = mailingListRepository;
    }

    public void Handle(OrderConfirmed orderConfirmed)
    {
        if (orderConfirmed == null)
        {
            throw new ArgumentNullException("orderConfirmed");
        }

        var order = orderConfirmed.Order;
        if (!order.ContactMe) return;

        var mailingListSubscription = new MailingListSubscription
        {
            Contact = order.PostalContact,
            Email = order.Email,
            DateSubscribed = DateTime.Now
        };

        mailingListRepository.SaveOrUpdate(mailingListSubscription);
    }
}

We simply implement the IHandle interface for a specific event, in this case OrderConfirmed. Because the handler is supplied by our IoC container, we can specify any services we require for the handler to do its job, in this case we require an IRepository<MailingListSubscription> in order for the handler to do its job.

There are similar handlers to send the confirmation email and create the new basket.

I’m not going to repeat Udi’s whole post here. Go and read it to see how the DomainEvent class is implemented. However I will repeat a crucial point: this is not about asynchronous messaging systems like MSMQ. The event pattern is simply a way to structure our software. When the order.Confirm() method is called, all the event handlers will fire synchronously before it returns.

Now there’s nothing to stop you from implementing an event handler that publishes an event onto an NServiceBus or MassTransit messaging system. Indeed, implementing domain events gives you the perfect hook for scaling your application as required by doing just that at a later date.

I really like this pattern, but there tend to be two concerns that people have when they encounter it:

1. Static methods suck! How can I test a static method buried in my domain entity? Yes, it’s not good OO practice to use static methods, but I think for certain infrastructure concerns, it’s OK. Here we have a single static method that can allow us to do some very nice refactoring, so I believe it’s a good payoff. Testability is not really an issue if you implement the DomainEvent class correctly.
2. It’s not clear what exactly is going to happen when I call order.Confirm(). Yes there is a level of indirection here, but that is a good thing. You can see that a domain event is being raised, and it’s pretty easy to find all the handlers, so I'd argue that it’s not really a problem.

If you want to see more detail, you can browse my commit for this change in the Suteki Shop source code: http://code.google.com/p/sutekishop/source/detail?r=415

How Conventions Can Massively Save Typing

Code is evil. It’s hard to understand, hard to get right, and hard to maintain. Every extra line of code you write is a bit of legacy that someone, probably you, is going to have to care about for as long as it’s used. Our first law as programmers should be to strive to write as little code as possible.

One of the things that really helps me write less code is conventions. By knowing that a particular thing will always be implemented in a certain way, we can write generic code that deals with the general case rather than the specific one. With Object Oriented languages like C# you do this with polymorphism. Build generic solutions around interfaces and follow conventions about how those interfaces are implemented.

I’ll show you some examples from my little eCommerce project, Suteki Shop.

The first convention I follow is to make every entity in my model implement the IEntity interface:

public interface IEntity
{
    int Id { get; set; }
}

It’s almost trivial, but it means that I know when a instance of an object is an entity and I know how to access its Id. A large proportion of my entities have a name property, so I generalise that too with INamedEntity:

public interface INamedEntity : IEntity
{
    string Name { get; set; }
}

For some of my entities, their order is important. They implement:

public interface IOrderable
{
    int Position { get; set; }
}

Others can be active or inactive:

public interface IActivatable
{
    bool IsActive { get; set; }
}

And the list goes on.

Now I know what is an entity and how to access its Id, it’s very easy to write a generic repository:

public interface IRepository<T> where T : class
{
    T GetById(int id);
    IQueryable<T> GetAll();
    void SaveOrUpdate(T entity);
    void DeleteOnSubmit(T entity);
}

We had an excellent debate about this a while back (check out the comments). I now would never claim this has anything to do with Domain Driven Development :)

Because I know if an entity is orderable I can write a generic orderable service:

public interface IOrderableService<T>
    where T : class, IOrderable
{
    IOrderServiceWithPosition<T> MoveItemAtPosition(int postion);
    int NextPosition { get; }
}

So now I can write generic code that persists entities to the database. I can easily write a generic controller, generic views, generic grids that show things in order. Because I know if an entity can be represented as an Id, Name pair, I can write generic code for drop down list boxes.

I start to get to the point where introducing a new entity is simply a question of defining its class. Development can proceed at a very rapid pace without writing much code.

The great thing about this approach is that if I want to create something more complex I can write it. I don’t have to use my generic controller I can create a specialised version if necessary.

Of course none of this is new, but it’s sometimes easy to forget how easy it is to leverage simple polymorphism to radically reduce the amount of code you need to write.

Implementing a ‘Money’ type in an ASP.NET MVC and NHibernate application.

Years ago I read Kent Beck’s seminal Test Driven Development. The first third of the book is a complete worked example of building a Currency type using TDD and I remember thinking at the time that this was an incredibly powerful technique. Rather than using basic data types -  int, string, DateTime - we could define what we actually meant. Rather than having a property Age of type int, we would define an Age type. Rather than two string properties we could define a Name type. One of the factors in moving Suteki Shop from Linq-to-SQL to NHibernate was NHibernate’s support for more complex finer-grained mapping.

When I first hacked together Suteki Shop, I used the decimal type everywhere I needed to represent money. This has mostly worked well for the simple scenario where a shop only has a single currency and that currency is sterling (£). But anyone wanting to sell in US dollars had to find every one of the many instances of “£” and replace them with “$”, and if you wanted to do something fancy with multiple currencies, you would have been out of luck. What I needed was a Money type. It’s not a trivial refactoring. Here are the steps I needed to take:

1. Create a Money type that behaved as far as possible like a .NET numeric type.
2. Create an ASP.NET MVC IModelBinder that knew how to bind the Money type.
3. Create an NHibernate IUserType that knew how to persist the Money type.
4. Change every point in the code that was using decimal to represent money to use the Money type instead.

Create a Money type

I wanted to create a Money type that would behave nicely when I did arithmetic with it. So I wanted to be able to write expression like this:

var x = new Money(43.5M);
var result = x * 3 + x/2 - 4;
result.Amount.ShouldEqual(148.25M);

To achieve this you have to write a lot of operator overloads for all combinations of operator, Money <-> Money, Money <-> decimal and decimal <-> Money. I won’t bore you, if you want to see the gory details, you can view the code here.

I also wanted my Money type to include the currency symbol when ToString was invoked, so I would get “£12.54” rather than “12.54”, but I found that this caused lots of problems with HTML forms and the default behaviour of the MVC HtmlHelper extensions. In the end I left ToString returning the number without the currency symbol and implemented a new method ToStringWithSymbol with it included. It feels like a horrible hack though.

I also overrode the equality operator, GetHashCode and Equals so make Money behave as expected in equality operations.

public class Money
{
    public decimal Amount { get; private set; }

    public Money(decimal amount)
    {
        Amount = amount;
    }

    public static string Symbol
    {
        get { return "£"; }
    }

    public static Money Zero
    {
        get { return new Money(0M);}
    }

    public override string ToString()
    {
        return Amount.ToString("0.00");
    }

    public string ToStringWithSymbol()
    {
        return Amount.ToString(Symbol + "0.00");
    }

    public override int GetHashCode()
    {
        return Amount.GetHashCode();
    }

    public override bool Equals(object obj)
    {
        var otherMoney = obj as Money;
        if (otherMoney == null) return false;
        return Amount.Equals(otherMoney.Amount);
    }

    public bool Equals(Money otherMoney)
    {
        if (otherMoney == null) return false;
        return Amount.Equals(otherMoney.Amount);
    }

    public static bool operator ==(Money a, Money b)
    {
        // If both are null, or both are same instance, return true.
        if (ReferenceEquals(a, b))
        {
            return true;
        }

        // If one is null, but not both, return false.
        if (((object)a == null) || ((object)b == null))
        {
            return false;
        }

        return a.Amount == b.Amount;
    }

    public static bool operator !=(Money a, Money b)
    {
        return !(a == b);
    }
    
    .....
}

Create an ASP.NET MVC IModelBinder

I wanted any of my forms that posted money values to work seamlessly when binding to entities with Money properties. For this to work I had to implement an IModelBinder for Money.

Creating the ModelBinder was pretty straightforward since I was just binding to a simple type.

public class MoneyBinder : IModelBinder
{
    public object BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
    {
        if (!typeof (Money).IsAssignableFrom(bindingContext.ModelType))
        {
            throw new SutekiCommonException(
                "MoneyBinder has attempted to bind to type '{0}', but may only bind to Money",
                                            bindingContext.ModelType);
        }

        var valueProviderResult = bindingContext.ValueProvider.GetValue(bindingContext.ModelName);

        try
        {
            var decimalValue = (decimal)valueProviderResult.ConvertTo(typeof(decimal));
            return new Money(decimalValue);
        }
        catch (Exception exception)
        {
            bindingContext.ModelState.AddModelError(bindingContext.ModelName, "Not a valid price.");
            return null;
        }
    }
}

But it could have been much easier. The IModelBinder interface doesn’t make it easy to work out how you are supposed to do simple binding. A SimpleModelBinder<T> with an abstract method like: T BindSimpleModel(string value) would have been nice.

The MoneyBinder is registered as a binder for the Money type in Global.asax.cs:

ModelBinders.Binders.Add(typeof(Money), new MoneyBinder());

Create an NHibernate IUserType

I wanted any entity that had a property of type Money to get automatically persisted by NHibernate as a numeric value. To do this I needed an IUserType. This tells NHibernate how to persist your custom type.

To create the NHibernate IUserType I borrowed Darrell Mozingo’s excellent BaseImmutableUserType<T>. With this it’s fairly straightforward:

public class MoneyUserType : BaseImmutableUserType<Money>
{
    public override object NullSafeGet(IDataReader rs, string[] names, object owner)
    {
        var amount = ((decimal?)NHibernateUtil.Decimal.NullSafeGet(rs, names[0]));
        return amount.HasValue ? new Money(amount.Value) : Money.Zero;
    }

    public override void NullSafeSet(IDbCommand cmd, object value, int index)
    {
        var moneyObject = value as Money;
        object valueToSet;

        if (moneyObject != null)
        {
            valueToSet = moneyObject.Amount;
        }
        else
        {
            valueToSet = DBNull.Value;
        }

        NHibernateUtil.Decimal.NullSafeSet(cmd, valueToSet, index);
    }

    public override SqlType[] SqlTypes
    {
        get
        {
            return new[]
                   {
                       SqlTypeFactory.Decimal
                   };
        }
    }
}

To register the MoneyUserType I just added a convention to my Fluent NHibernate setup:

public class FluentNHibernateConfigurationBuilder : IConfigurationBuilder
{
    ....

    public static void ConfigureMappings(MappingConfiguration mappingConfiguration)
    {
        mappingConfiguration.FluentMappings
            .AddFromAssembly(typeof (ProductMap).Assembly)
            .Conventions.Add(
                ForeignKey.EndsWith("Id"),
                PrimaryKey.Name.Is(x => x.EntityType.Name + "Id"),
                DefaultCascade.None(),
                new MoneyConvention());
    }
}

public class MoneyConvention : UserTypeConvention<MoneyUserType>{}

Change every point in the code that was using decimal to represent money to use the Money type instead

I had naively hoped that a good enough Money class would allow me to just change the type of money properties from decimal to Money and have done with it. But even with the operator overloading I found that there were many places where I needed to add ‘.Amount’ to make the code behave. I could have implemented an implicit cast, but I don’t like not knowing where types are being converted. I’ve found in the past that it can lead to subtle, hard to find, bugs.

What I have now is a single point where I can change the way money behaves in Suteki Shop. Providing multi-currency or changing the behaviour of money in any way should be far easier. And if you simply want to take Suteki Shop and sell in Dollars, Yen or Euros, it’s now just a single line change to the code. And yes, configurable currencies will come very soon.

Some thoughts on the C# type system

This was far more work than it needs to be and is a clear example of the deficiencies of the C# type system. With Haskell’s type classes or Scalar’s implicits it would have been possible to create a new numeric type without having the change ripple through the entire application. The Scalar implicit is a very interesting idea and would surely be a very cool addition to the C# type system. Anders?

Getting the Property Name From a Property Expression

Here’s a common problem. Say we have a function that returns the name of a property from a property expression:

public static string GetPropertyNameFromPropertyExpression<T>(Expression<Func<T, object>> propertyExpression)

We might use it like this:

var propertyName = GetPropertyNameFormPropertyExpression<Grandparent>(g => g.Parent.Child.Name);
Console.WriteLine(propertyName);

Which should output:

Parent.Child.Name

The problem splits itself nicely into three parts:

1. Get each chained property name.
2. Intersperse with some separator, in our case “.”
3. Concatenate.

We’ll do the hardest part first, recursively walking down the expression, yielding PropertyInfos as we go:

public static IEnumerable<PropertyInfo> GetProperties<T>(Expression<Func<T, object>> propertyExpression)
{
    return GetProperties(propertyExpression.Body);
}

private static IEnumerable<PropertyInfo> GetProperties(Expression expression)
{
    var memberExpression = expression as MemberExpression;
    if (memberExpression == null) yield break;

    var property = memberExpression.Member as PropertyInfo;
    if (property == null)
    {
        throw new SutekiCommonException("Expression is not a property accessor");
    }
    foreach (var propertyInfo in GetProperties(memberExpression.Expression))
    {
        yield return propertyInfo;
    } 
    yield return property;
}

This will return an list of properties for ‘Parent’, ‘Child’ then ‘Name’. It uses the classic recursion pattern of a guard clause (testing for a null MemberExpression) followed by the recursive call. It also shows that you can use the ‘yield return’ syntax recursively. It’s a shame there’s not a ‘yield many’ statement, then we wouldn’t have to do the foreach loop and trigger the enumeration.

We can get the names of the properties by selecting the PropertyInfo.Name:

GetProperties<Grandparent>(g => g.Parent.Child.Name).Select(property => property.Name)

Next we need a function to intersperse our “.”. We can write this as a very generic Intersperse:

public static IEnumerable<T> Intersperse<T>(this IEnumerable<T> items, T separator)
{
    var first = true;
    foreach (var item in items)
    {
        if (first) first = false;
        else
        {
            yield return separator;
        }
        yield return item;
    }
}

Last of all we need a 'Concat’ function to concatenate the strings together:

public static string Concat(this IEnumerable<string> items)
{
    return items.Aggregate("", (agg, item) => agg + item);
}

Now we can fill in the body of our GetPropertyNameFromPropertyExpression function:

public static string GetPropertyNameFromPropertyExpression<T>(Expression<Func<T, object>> propertyExpression)
{
    return GetProperties<Grandparent>(g => g.Parent.Child.Name)
        .Select(property => property.Name)
        .Intersperse(".")
        .Concat();
}

I’m becoming a big fan of a more functional style of programming. Granted it’s not as efficient as the imperative version, but I really like the elegance. Anyone who’s looked at Haskell will recognise the ‘Intersperse’ and ‘Concat’ functions, although my crappy C# versions are rather blunt instruments compared with the beauty of Haskell.

Trust Your Instincts! Or how to stay interested in programming

I just read a blog post by John Sonmez, Keep Pressing on my Friend (When programming seems bleak). It’s all about the lack of motivation that hits everyone at one time or another.

“Do you ever get the feeling that you just want to get a cabin in the woods and never see a computer again?”

John’s advice is to ‘keep pressing on’, but I don’t agree. I think forcing yourself to do something that your mind is rebelling against is just going to make the burn-out worse long term. Maybe you should just go to the cabin in the woods? After a week or two you’d probably be itching for a programming challenge.

My advice is different: follow your instincts. Keep your child-like curiosity alive and don’t force yourself to learn something you’ve no inherent interest in.

I’ve got two small children, Leo 8 and Yuna 2. They are always curious, always wanting to know why things are the way they are, and what this or that is. They don’t get burn-out. But they are quite picky about what they are interested in learning. I can’t insist that they pay attention to any particular thing, and a lot of things I’d like them to care about, they simply don’t. It’s as if they have some inner guide telling them what is important and what is not. But when something does intrigue them, they show total concentration.

When something really intrigues me, I’m the same. There’s no effort involved in forcing myself to learn more about it. It’s the opposite, it’s a pleasure. The problem is that I can’t make myself interested in something. I can’t get intrigued by WCF simply because I think I ought to be learning it. Forcing myself to learn something I have no particular interest in is slow and painful. Even if I do struggle through, I find it hard to retain what I’ve learnt.

I’ve come to the conclusion that there is simply no point struggling with stuff I’m not interested in. It’s just not worth it. In any case, if I don’t find something interesting, why would I want to be doing it in the first place? I’m now trying a new learning technique. I call it ‘wow, that’s cool’ focused learning. Regardless if anything is necessary for professional development, or other grown up reasons, I’m simply going to focus on stuff that grabs me.

For example, for the last couple of months I’ve been working (very) slowly through Bryan O’Sullivan’s ‘Real World Haskell’, not because I think that I’ll get a better daily rate if I put it on my CV, but because Haskell is an awesomely different way of thinking about programming. Is it ‘better’ or ‘more useful’? Will it ‘work in corporate IT’? Will the Morts understand it? Is it relevant in any way? I don’t care. I think it’s cool and that’s all matters.

Programming is the best job in the world. You get to play on the cutting edge of human culture, making the future up as you go. It can never be boring. Don’t let ‘serious’ considerations get in the way of the pure intellectual joy it can give you.

More Boilerplate Code Removal

In my quest to never repeat myself, here’s another little trick that I use.

Say we have a number of operations represented by methods that all take the same parameter. Here’s the static data creation code in Suteki Shop again:

public void Insert1()
{
    var sessionFactory = configuration.BuildSessionFactory();
    using (var session = sessionFactory.OpenSession())
    using (var transaction = session.BeginTransaction())
    {
        InsertRoles(session);
        InsertAdministrator(session);
        InsertRootCategory(session);

        transaction.Commit();
    }
}

It’s really boring that we have to repeatedly write ‘(session)’. Is there anther way? Oh yes! Enter my ‘ApplyTo’ extension method:

public static void ApplyTo<T>(this T arg, params Action<T>[] actions)
{
    Array.ForEach(actions, action => action(arg));
}

Now our Insert method looks much cleaner:

public void Insert()
{
    var sessionFactory = configuration.BuildSessionFactory();
    using (var session = sessionFactory.OpenSession())
    using (var transaction = session.BeginTransaction())
    {
        session.ApplyTo(
            InsertRoles,
            InsertAdministrator,
            InsertRootCategory
            );

        transaction.Commit();
    }
}

Isn’t that nicer?

Adventures in Repetitive Code

One of the things you often see in legacy systems is repetitive little blocks of code. Take this example from Suteki Shop, here we’re inserting some static data into the database to setup the initial set of roles:

static void InsertRoles1(ISession session)
{
    var admin = new Role { Id = 1, Name = "Administrator" };
    session.Save(admin);
    
    var orderProcessor = new Role { Id = 2, Name = "Order Processor" };
    session.Save(orderProcessor);
    
    var customer = new Role { Id = 3, Name = "Customer" };
    session.Save(customer);
    
    var guest = new Role { Id = 4, Name = "Guest" };
    session.Save(guest);
}

The only thing that varies in these four code blocks is the id and the name. One way of factoring this out would be to write a little nested private class:

private class RoleInserter
{
    private readonly ISession session;

    public RoleInserter(ISession session)
    {
        this.session = session;
    }

    public void Insert(int id, string name)
    {
        var role = new Role { Id = id, Name = name };
        session.Save(role);
    }
}

So our InsertRoles method now looks like this:

static void InsertRoles1(ISession session)
{
    var roleInserter = new RoleInserter(session);

    roleInserter.Insert(1, "Administrator");
    roleInserter.Insert(2, "Order Processor");
    roleInserter.Insert(3, "Customer");
    roleInserter.Insert(4, "Guest");
}

But now we’ve actually got more code in total than before. Also it’s one of the unfortunate things about object oriented programming that simple utility functions like this are awkward. The silly name of the class ‘RoleInserter’ is a dead giveaway. Steve Yegge calls it The Kingdom of Nouns. We don’t really need a whole class, we just need a single function.

Let’s use a closure to replace RoleInserter with a function:

private static Action<int, string> GetRoleInserter(ISession session)
{
    return (id, name) =>
    {
        var role = new Role { Id = id, Name = name };
        session.Save(role);
    };
}

Now our InsertRoles method looks like this:

static void InsertRoles1(ISession session)
{
    var insertRole = GetRoleInserter(session);

    insertRole(1, "Administrator");
    insertRole(2, "Order Processor");
    insertRole(3, "Customer");
    insertRole(4, "Guest");
}

There’s little need to factor out the GetRoleInserter, it’s simpler just to write it in line:

static void InsertRoles(ISession session)
{
    Action<int, string> insertRole = (id, name) =>
    {
        var role = new Role {Id = id, Name = name};
        session.Save(role);
    };

    insertRole(1, "Administrator");
    insertRole(2, "Order Processor");
    insertRole(3, "Customer");
    insertRole(4, "Guest");
}

That’s much nicer. Using lambdas like this can really clean up little repetitive code blocks.

Using the Visitor Pattern with Domain Entities

Here is an interesting problem I was faced with recently while working on a Case Management system. The core entity of this system is the Case, which represents a group of people working on a single incident or investigation. Each case is made up of a number of Activities, and these in turn are made up of a number of Tasks. A requirement of the system is that Cases, Activities and Tasks can all have notes attached to them. When the user asks for the notes against a case they want to also see all the notes attached to the case’s activities and tasks. The model looks like this:

There were other requirements that also needed us to collect information from the entire aggregate. The functionality to navigate the aggregate - to go from a case to its activities and then to its tasks - needed to be separate from whatever we needed to do at each entity. In the notes case, we wanted to collect all the notes, but when calculating the time taken on a case, we wanted to add up all the time taken on each individual task.

This is a perfect scenario for the visitor pattern. It is designed to separate out the navigation of relationships from the action that needs to be performed at each node.

All our domain entities implement a layer supertype called Entity. We added the ability for the entity to accept a visitor and for child collections to be registered:

public interface IEntity
{
    TVsistor AcceptVisitor<TVsistor>(TVsistor visitor) where TVsistor : IDomainVisitor;
}

public class Entity : IEntity
{
    private IEnumerable<IEntity> childEntities;

    public TVsistor AcceptVisitor<TVsistor>(TVsistor visitor) where TVsistor : IDomainVisitor
    {
        visitor.Visit(this, VisitChildren);
        return visitor;
    }

    protected void RegisterChildCollection(IEnumerable<IEntity> childCollection)
    {
        childEntities = childEntities == null ? childCollection : childEntities.Concat(childCollection);
    }

    private void VisitChildren(IDomainVisitor visitor)
    {
        if (childEntities == null) return;
        foreach (var childEntity in childEntities)
        {
            childEntity.AcceptVisitor(visitor);
        }
    }
}

The Domain Visitor interface is very simple, a Visit method accepts an entity to visit and a visitChildren delegate so that the visitor can optionally visit the entity’s children in whatever order it wants:

public interface IDomainVisitor
{
    void Visit(IEntity entity, Action<IDomainVisitor> visitChildren);
}

Here is our domain model, notice that we register Activities as a child collection of Case and Tasks as a child collection of Activity. We could also have registered each Notebook’s Notes collection as well, but so far there’s no requirement to include notes in any of the traversals of case. Note also the Notes property of Case. It creates and accepts a NotebookVisitor (see below) that collects all the notes from the graph:

public class Case : Entity, IHaveNotes
{
    private readonly IList<Activity> activities = new List<Activity>();
    public IList<Activity> Activities
    {
        get { return activities; }
    }

    public Case()
    {
        RegisterChildCollection(Activities);
    }

    public Notebook Notebook { get; set; }

    public IEnumerable<Note> Notes
    {
        get
        {
            return AcceptVisitor(new NotebookVisitor()).Notes;
        }
    }
}

public class Activity : Entity, IHaveNotes
{
    private readonly IList<Task> tasks = new List<Task>();
    public IList<Task> Tasks
    {
        get { return tasks; }
    }

    public Activity()
    {
        RegisterChildCollection(Tasks);
    }

    public Notebook Notebook { get; set; }
}

public class Task : Entity, IHaveNotes
{
    public Notebook Notebook { get; set; }
}

public interface IHaveNotes
{
    Notebook Notebook { get; set; }
}

public class Notebook
{
    private readonly IList<Note> notes = new List<Note>();
    public IList<Note> Notes
    {
        get { return notes; }
    }
}

public class Note
{
    public string Text { get; set; }
}

Finally here’s the NotebookVisitor, it simply checks if the currently visited entity has notes and then appends them to its notes collection:

public class NotebookVisitor : IDomainVisitor
{
    public IEnumerable<Note> Notes { get; private set; }

    public void Visit(IEntity entity, Action<IDomainVisitor> visitChildren)
    {
        var iHaveNotes = entity as IHaveNotes;
        if (iHaveNotes != null)
        {
            Notes = Notes == null ? iHaveNotes.Notebook.Notes : Notes.Concat(iHaveNotes.Notebook.Notes);
        }
        visitChildren(this);
    }
}

Here’s a test that shows all this working:

public void NotebookVisitor_visits_all_notes_in_graph()
{
    Func<string, Notebook> createNotebook = text => new Notebook
    {
        Notes = {new Note {Text = text}}
    };

    Func<string, Task> createTask = text => new Task
    {
        Notebook = createNotebook(text)
    };
    
    var @case = new Case
    {
        Activities =
            {
                new Activity
                {
                    Tasks =
                        {
                            createTask("Task 1 note"),
                            createTask("Task 2 note")
                        },
                    Notebook = createNotebook("Activity 1 note")
                },
                new Activity
                {
                    Tasks =
                        {
                            createTask("Task 3 note"),
                            createTask("Task 4 note")
                        },
                    Notebook = createNotebook("Activity 2 note")
                }
            },
        Notebook = createNotebook("Case 1 note")
    };

    foreach (var note in @case.Notes)
    {
        Console.WriteLine(note.Text);
    }
}

Which prints out:

Case 1 note
Activity 1 note
Task 1 note
Task 2 note
Activity 2 note
Task 3 note
Task 4 note

If you have lots of repeated code in your application that has loops within loops navigating down object graphs in order to collect some information or perform some action, take a look at the visitor pattern, it may save you a lot of typing.

How to Fix Public Sector IT

As the Chinese curse has it, ‘May you live in interesting times’. These are certainly interesting times for the UK’s public sector. The coalition government has stated that it intends to cut non-health public services by 25 to 40% - the biggest cuts since the Second World War. They also say that most of these cuts can be had by efficiency savings. I personally think that they can cut that much from public sector IT and leave it healthier, but it needs to be done in the right way.

In the private sector money rules. Capitalism is an ongoing process of Schumpeterian creative destruction as firms are created, compete for profits and die. Survival depends on firms cost-cutting and innovating. The public sector exists in an entirely different context. How do you create a public sector framework that incentivises efficiency and encourages innovation, and more importantly, allows for failure?  This has been a question for every government since the invention of taxes.

Well, there is an evolving self-organising system, not based on a monetary economy that often beats Capitalism at its own game. It produces more variety, better quality and out-innovates. It is, of course, the world of open-source software (OSS), one of the most profound and utterly unexpected outcomes of the internet. What is surprising is that it hasn’t been studied in more detail by policy wonks. I believe that public sector organisations could be transformed by learning lessons from OSS, and this especially applies to public sector IT.

Here’s what the government should do:

Every business is a software business

A common refrain you hear from managers, not just in the public sector, but they say it more than others, is, “We are not a software business”. Wrong. Every business in the 21st century is a software business. Much of the work of the public sector is information management: tax collection, benefits, regulation, health records, the list goes on. This work has IT at its core.

Unfortunately the prevailing trend has been to outsource software development and infrastructure. The UK civil service has gone from being a focus of expertise to being largely hollowed out. This needs to change. The civil service needs to put expertise in IT at its core. In the same way that the government is putting doctors back in charge of the NHS, it needs to put geeks in charge of IT.

Build IT Skills in the civil service

The civil service should work hard at becoming a major centre of UK IT skills. That means bringing the majority of IT expertise back in house and fostering a culture that makes the civil service a place where geeks want to work. Simple things like dress codes are important. There is simply no reason why a programmer should not be allowed to come to work in a t-shirt, shorts and flip-flops. In fact there’s no real reason for them to come to work at all – there is no need for people to be physically co-located in 2010. Insisting on this severely limits the pool of candidates. Public sector managers need to understand what motivates knowledge workers and handle them appropriately.

There also has to be a career path for developers that allows great coders to stay being great coders and not have to become managers. The UK IT workforce is warped. Because coding skills are undervalued, salaries tend to be low compared to jobs that require a similar level of expertise. This means that programmers almost always join the contract market at some point in their career, simply because it better reflects the real supply and demand for their skills. Many teams reflect this with the junior members being predominantly permanent staff and the senior ones being contractors. This is a problem, because permanent staff in are almost impossible to fire. When cost cutting is demanded, the contractors are laid off first. This guts the organisation of its core skills.

So pay rates for programmers need to be higher. But how do you know if a programmer is good enough to be paid these new high rates? There are no realistic qualifications in IT. I believe that a reputational system based on OSS is the best way to judge a programmer’s quality. It seems natural that a contributor to a successful, high-profile system should be valued more than someone without any demonstrable experience. Open-sourcing government IT would automatically build a transparent reputational market for IT skills.

Fix procurement

IT procurement in the public sector is broken. The seemingly laudable aim of regulating and monitoring procurement, especially of high value contracts has put a huge barrier in front of businesses that wish to supply the public sector. This distorts markets and encourages rent seeking behaviour. Only larger companies with deep pockets are able to afford the lengthy process of bidding for public sector contracts. And of course, once you’ve spent millions on securing a contract, you expect some payback. This leads to hugely inflated costs for software. Not only that, but the main effort and expertise of suppliers is focused on pre-sales and contract negotiation, once the contract is signed you often find that more junior managers and technicians take over.

As a freelance developer I have been hired several times by large suppliers to work on teams developing software for the public sector. There is no secret ingredient that they have. Often the project management and in-house development skills are poor to say the least. In many cases the team building the software is almost entirely freelancers, in which case the end-client could have hired them directly at a fraction of the cost, and probably experienced less friction in the process.

The outsourcing trend means that civil servants with little software development experience themselves have to manage contractual relationships with suppliers. They don’t know what questions to ask. They often have little idea of the cost of particular features or applications and have been acclimatised by suppliers to expect large numbers.  Before long you have the situation where a few tens of thousands of pounds for some fields to be changed seems perfectly reasonable. I kid you not, I’ve seen it with my own eyes.

It’s not only bespoke software that suffers this pathology, it almost always applies to large COTS (Commercial Off The Shelf) procurements too. They are almost never wholly ‘off the shelf’, there’s always some degree of customisation required and the cost of customising a shrink wrapped product is always more than modifying your own internally built software.

Embrace open source

Because of the lack of internal IT skills and the corresponding reliance on external suppliers, there is a very negative attitude to open source software in the public sector. Of course you, dear reader, know full well that many open source software products are not only much cheaper (as in free) than their commercial equivalents, but are also often better quality. But without the internal expertise to evaluate them your only option is to listen to salesmen from commercial suppliers, and that is pretty much how most products get chosen in the public sector.

A civil service skilled in IT would be able to not only evaluate open source products, but also contribute to them and instigate new ones. The network effects would be significant and would have the potential to turn the UK into a world leader in open source development. Applications, infrastructure and innovations could be far more readily shared between departments, but it would also be a way for tax pounds to directly contribute to wider economy. Indeed, it only seems fair that a product produced at the taxpayer’s expense should be available to the taxpayer to use.

It should be policy to only consider a commercial product if it can be demonstrated that no practical open source alternative exists. Where there is no existing open source alternative, the first consideration should be to extend an existing project or create a new one from scratch. All software developed by the public sector should be open source by default. Only where there are demonstrable security concerns should this not be done.

By building software in the open, public sector organisations will be able to learn from each other in a real and direct way. This means being public about failures as well as successes. In the same way that I know the names of the developers of NHibernate and the Castle Project (two very successful open source projects), the names and reputations of the technologists and managers behind successful public sector projects will also be well known. When an organisation is considering a major new build they will know who to turn to, not on the basis of slick presentations, but on real measurable results.

An example of successful in-house development

I’ve just finished a very pleasant year working as a technical-architecture consultant with the UK’s Pensions Regulator (TPR) which is part of the Department for Work and Pensions. It is public knowledge that this department has been asked to prepare estimates for 25% to 40% cuts.

TPR’s work as a regulator is predominantly casework based. They monitor UK private sector pensions, risk-assess them, and provide advice to trustees. The case management process has been handled historically using a combination of off-the-shelf software products, spreadsheets and word documents. A dedicated case-management system was desperately needed.

Now you might think that you can buy a case-management system off the shelf, and yes, you can. But you can’t buy one specialised for managing pension regulation. Any product would have to be deeply customised. It would also need to be integrated with the rest of TPR’s systems to be effective. Some effort was expended looking at commercial alternatives, most with costs of around £100,000+. This does not include the customisations, which would, of course, have to be carried out by the supplier. The cost of the consultancy work could only be estimated from detailed specifications, the creation of which would have imposed considerable extra cost to TPR. There is little doubt that the internal and consultancy costs would have easily outweighed the ticket price.

But luckily TPR, for historical reasons, has a strong internal software development team. So after some debate, it was decided to build a bespoke case-management system from scratch. This took a team of two developers around 7 weeks to complete with a conservatively estimated cost of around £30,000. That’s right, building bespoke software in-house was at most 30% of the cost of using an external supplier. Because the system was built in-house there was no need for lengthy contract negotiations and detailed specifications, instead we built it incrementally with constant input from the case management team. They got exactly what they needed at a fraction of the cost.

Going forward, there won’t be any need for expensive maintenance contracts and huge costs for each change or refinement, because it can all be done in-house. Of course TPR need to maintain their development team, but I firmly believe that those guys are a lot cheaper than the outsourced alternative.

It’s a piece of software that I was very proud to have been a part of. It would have been useful to other public (and private) sector organisations, not simply as a casework system, but as a demonstration of a particular technology stack and methodology. It would also have been an excellent advertisement for the technical prowess of the team. It’s a real shame that this great success story can’t be told outside the confines of TPR. You will never get to see the beautifully factored code with over 1500 unit tests, or know the names of the people who put it together.

So, Mr Cameron, here’s something truly radical you could do that fits wholeheartedly with your aims for a more open and inclusive government:

open source it!

The New Brighton ALT.NET Wiki

I’ve just created a wiki, using the excellent Screwturn Wiki, for the monthly Brighton ALT.NET Beers:

http://brightonalt.net/

If you’ve attended in the past, or plan to in the future, or even if you’re a Brighton based dev, please create an account for yourself and add your name to the list of members with a little info about yourself too.

It would also be very nice if we could update the wiki with stuff that we talk about, maybe a new page for each meeting.