Friday, April 11, 2008

MVC Storefront

Rob Conery, the genius behind Subsonic, and now assimilated by the Borg, has just started a series of posts describing the process of building an eCommerce application using the MVC Framework. What's really cool about it is that he's soliciting input from some well known figures in the community as he evolves the project, so in the first screencast of the series he talked to Ayende and Steve Harman about some initial architectural choices including the repository pattern.

So far there are three episodes:

ASP.NET MVC: Introducing The MVC Storefront Series

ASP.NET MVC: MVC Storefront, Part 2

ASP.NET MVC: MVC Storefront, Part 3

And the source is available on Codeplex here.

The brilliant thing is, I'm currently working on an MVC eCommerce application as well. It's called sutekishop. It has a real first customer who's actually paying for it, but it's also open source. I've set up a Google code project here:

http://code.google.com/p/sutekishop/

I'm going to be keeping a very close eye on Rob's progress and incorporating as many of his ideas as possible into my project. So far it's very early days and I've only got a repository and the IoC container set up, but I'll be posting on my progress, so watch this space.

Thursday, April 10, 2008

Do Not Use Webhosting.uk.com

Disclaimer: I'm very reluctant to rubbish a company in public like this, but I think in this case the service has been so awful I feel a duty to warn others about them.

For the last few months I've been hosting a personal web site at webhosting.uk.com. I chose them after a friend recommended them for being relatively cheap and flexible. It sounded like a good deal, you could have as many sites as you liked within certain limits of disk space and bandwidth.

One morning I noticed the web site was down, but since it's not business related I didn't follow it up until later in the day. They provide an online technical support chat, so I fired it up and was soon told that there was a problem with some files being corrupted on the server but service would be resumed in six hours, i.e. later than night. The next morning the site was still unavailable. Once again I was given a figure of six hours and told not to worry because my data was safe. That evening the site was still unavailable. They continually failed to provide honest information about the state of the server. It took a further two days before they finally admitted that they had lost the drive and had no backups:

"1.The Windows OS got corrupt. That was the reason we decided to reinstall it. 2.This was a RAID server. To ensure that there is no data loss, we had multiple drives on the server wherein the web-date, MySQL & MS-SQL were backed up separately. This was done on the same server but on different drives. 3.While we were checking the server during OS re-installation, we also found that the primary hard-drive had faults on it and was not responding. Further evaluations and investigations revealed that the drive itself was faulty. Since this was a part of the RAID array, we had to deal with it very carefully. One single error would have resulted in the data getting wiped out. Inspite of all efforts, the data on the drive was not getting retrieved. That was the reason for the delays."

It's really hard to understand what happened from this garbled explanation. On the one hand they say they were using a RAID array, but then they contradict this by implying that different things were stored on different disks: "we had multiple drives on the server wherein the web-data, MySQL & MS-SQL were backed up separately. This was done on the same server but on different drives". That doesn't sound like my understanding of RAID.

But in any case, simply backing stuff up on a separate disk in the same box is a pretty crap strategy. Also simply doing database backups isn't sufficient. My site allowed my users to upload photos and these have all been lost. I've also lost the (somewhat ancient) code for the site since it was developed a long time ago on a long dead machine. Maybe I was naive to expect a host to back things up.

Since then the only compensation they've offered has been three month's of free hosting, even though I've probably lost at least £1000 in development time (to rewrite the software) plus the hours I wasted chasing them up plus the irretrievable loss of my user's photos. Free hosting is completely worthless from a company I've lost all trust with.

So, I'm now looking for a good hosting company. I'm quite interested in getting a dedicated server that I can look after myself. Any recommendations?

Monday, April 07, 2008

IoC Container Purse Fight

There's a great discussion of the problems and benefits of IoC containers on Serial Seb's blog: http://serialseb.blogspot.com/2008/04/ioc-container-is-not-carpet-it-floor.html

Thursday, April 03, 2008

Repository.GetById using LINQ Expression Syntax

A while ago I talked about using the IRepository<T> pattern with LINQ to SQL. One of the methods of my repository is GetById. The slight difficulty here is that we need to discover the primary key property of the (generated) type at runtime so we can't use a vanilla lambda expression in the Where clause. Before I used the DynamicQueriable helper library, but I've just re-written the function using an explicit expression tree which removes the need to reference the helper library.

public T GetById(int id)
{
    var itemParameter = Expression.Parameter(typeof(T), "item");

    var whereExpression = Expression.Lambda<Func<T, bool>>
        (
            Expression.Equal(
                Expression.Property(
                    itemParameter,
                    typeof(T).GetPrimaryKey().Name
                ),
                Expression.Constant(id)
            ),
            new ParameterExpression[] { itemParameter }
        );

    return dataContext.GetTable<T>().Where(whereExpression).Single();
}

Here is the extension method that finds the primary key:

public static PropertyInfo GetPrimaryKey(this Type entityType)
{
    foreach (PropertyInfo property in entityType.GetProperties())
    {
        ColumnAttribute[] attributes = (ColumnAttribute[])property.GetCustomAttributes(typeof(ColumnAttribute), true);
        if (attributes.Length == 1)
        {
            ColumnAttribute columnAttribute = attributes[0];
            if (columnAttribute.IsPrimaryKey)
            {
                if (property.PropertyType != typeof(int))
                {
                    throw new ApplicationException(string.Format("Primary key, '{0}', of type '{1}' is not int",
                        property.Name, entityType));
                }
                return property;
            }
        }
    }
    throw new ApplicationException(string.Format("No primary key defined for type {0}", entityType.Name));
}

Extension Properties for easy Reflection in C#

It's not generally considered good practice to define extension properties on object. But there's one scenario where I've found that it can be really nice. I've been copying the pattern of using anonymous types as dictionaries from the MVC Framework as described here by Eilon Lipton. This means you have to reflect over the anonymous type's properties to enumerate the dictionary. I've factored this reflection into an extension method for class:

public static IEnumerable<NameValue<string, object>> GetProperties(this object item)
{
    foreach (PropertyInfo property in item.GetType().GetProperties())
    {
        yield return new NameValue<string, object>(property.Name, () => property.GetValue(item, null));
    }
}

public class NameValue<TName, TValue>
{
    public TName Name { get; private set; }
    public TValue Value { get { return valueFunction(); } }

    Func<TValue> valueFunction;

    public NameValue(TName name, Func<TValue> valueFunction)
    {
        Name = name;
        this.valueFunction = valueFunction;
    }
}

Note that by using a lambda as the value in the NameValue class we defer actually accessing the value of the property until we need it.

Here's a test showing how you would use GetProperties:

[Test]
public void GetPropertiesShouldReturnThePropertiesOfAnAnonymousType()
{
    var item = new { Message = "Hello World", Number = 4 };

    Assert.AreEqual("Message", item.GetProperties().First().Name);
    Assert.AreEqual("Hello World", item.GetProperties().First().Value);
    Assert.AreEqual("Number", item.GetProperties().ElementAt(1).Name);
    Assert.AreEqual(4, item.GetProperties().ElementAt(1).Value);
}

Tuesday, April 01, 2008

Software Factories and Battery Programmers

batteryHen

A while back I had a client who was moving to a Web Service based ESB architecture. They had a number of Visual Basic 6 developers that they wanted to bring up to speed developing .NET web services and at the time I was getting interested in software factories, especially the P&P group's GAT. Their lead architect suggested that I write a software factory based on a standard web service architecture. Her intention was that she could get her VB devs to build web services without having to have a deep level of understanding about how they work. I had a great deal of fun building it, even though it was very early days for the GAT, and I learnt a lot about component based development by reading the P&P code. These days I look back on that job as one of my most enjoyable.

But how would I feel if I was one of those devs asked to use my GAT product? I'd hate it. Where's my joy? Everything I love about programming would have already been done by someone else. Not only that but I couldn't learn any lessons or move the architecture forward because it's all been locked down.

Every time I start a new project I tend to architect it in a different way. I learn new stuff all the time, recently it's been IoC containers, MVC frameworks and LINQ. If I was somehow constrained to do things the same way I would be far less productive and happy than I am (and I'm not that productive and happy in any case :P). The industry is too young, too fluid and too diverse to allow you to lock up your architecture without paying an awful price in lost opportunities.

I don't want to go into it now, but I don't like the architectural patterns imposed by a lot of the publicly released software factories from the P&P group. I think data transfer objects are a total waste of time in 90% of applications for a start. I would be mightily depressed if I had to work in a shop that had adopted them... oops, I did go into it...

And what about those legacy skilled VB developers. Well, you can guess... the software factory didn't allow them to quickly build web services without understanding the basics about .NET, XML and SOAP. They needed extensive training and hand holding. There's no replacement for skilled developers no matter what the tool venders tell you.

So Software Factories will go the same way as CASE tools, visual tools and code generators. After wasting lots of people's time and money they'll be cast into the void of software evolutionary dead ends.

Visual Tools: Marketing Dream, Programmer's Nightmare

I really enjoyed reading Secret Geek's (AKA Leon Bambrick) post today: Workflow software: I'm calling the bluff. Leon has a major gripe about workflow tools, especially the graphical tools that are their real selling point. Non-programmers love graphical tools. Why? Because code scares them, whereas the pretty pictures seem to make so much more sense. Non only that but, wonder of wonders, you can get business users to design the software. Wow, think of the savings of removing those expensive, scruffy awkward coders. Leon throws down a challenge:

"Show me a working 'business analyst' -- one, who is not now nor has ever been a coder -- who successfully designs 'business workflows' using an off the shelf tool, and who didn't require *any* expensive training, and who achieves their task in less time and with more precision than a coder. And who doesn't need to call technical support for help at the time.

Show me just one.

World wide.

I can wait. I give you one month. Nah, screw it. I give you eternity."

Somehow, I don't think he's going to see any challengers.

Visual tools make for great marketing. That's why so many companies spend so much money on Enterprise Workflow/Service Bus/Business Rule Engines. It's simple evolutionary psychology, Humans are evolved from fruit eating primates, we instinctively like anything with lumps of fruity colors. Code doesn't have that, it looks like something scary and mathematical. It's very hard to sell. But show some management type a colorful tool where you can simply drag and drop a few shapes to make a simple workflow and they're sold.

So what's the problem? It's simple: software is complex. Now that's not just a bit tricky, like sodoku say; it's seriously complex. Most visual tools give you little boxes that do an operation like a decision or maybe a calculation and those boxes are joined together with little lines. Hey, it's just like a programming language, except prettier. The problem is it's just like the simplest programming language. In fact it's a bit like the orginal BASIC:

BiztalkEqualsTRS80

I used to write very simple BASIC programs on my TRS-80 as a child. But even they sometimes had hundreds of lines of code. Imagine a picture with a hundred little boxes and several hundred little lines connecting them. You can only see the whole thing when it's zoomed too small to read any of the captions, and if you can read the captions you'll only see at best a dozen boxes.

The thing is, we've moved on a little since TRS-80 BASIC. Modern programming languages are fundamentally the distillation of fifty years of lessons learnt; pretty much every language innovation: loops, conditionals, stacks, functions, and all the technology of object oriented programming, not to mention the functional stuff, is there to help us manage the awesome complexity. There are hundreds of books and a million web pages written on the subject: algorithms, object orientation, design patterns, refactoring, testing, the list is endless. A good programmer has to work really hard to keep up with all this stuff. But the result is that with the current state of the art we can do a reasonable job at keeping this horrendous complexity under control.

If you use a visual programming tool, you're throwing all this away. You cannot write anything other than a simple demo program with a visual tool. If you try you will fail.

There's one place where I think visual tools succeed and that's when they're used to represent static structure. I would much rather look at a data model in SQL Server's Database Diagram tool than trying to grep a long list of create table statements, and of course, it's far easier to work on a user interface with a graphical designer than by hand coding the construction and mapping of loads of objects. I actually rather like the class designer in Visual Studio. But I will never use a graphical tool to create program flow or logic.

Wednesday, March 26, 2008

I'm speaking at the next Sussex Geek Dinner

Simon Harriyott has invited me to talk at the next Sussex Geek Dinner on Wednesday, April 23, 2008 at 8:00  PM at The Black Horse on Church Street in Brighton.  Here's a map.


View Larger Map

I'll be reprising my DDD6 talk, "Why do I need an Inversion of Control Container", but renamed as "Alternative Architectures: Inversion of Control", since Simon thought the first title was too geeky ;) I've covered a lot of the subject matter here if you'd like an introduction. Check out:

The Castle Project's Windsor Container and why I might need it.

What is Inversion of Control

Windsor Container Resources

Come along if you can, it should be fun. Sign up here.

Tuesday, March 25, 2008

File uploads with the MVC Framework

UPDATE: This post is out of date. It talks about an earlier version of the MVC Framework. Please check the ASP.NET MVC documentation for how to do file uploads.

There have been a couple of questions on the ASP.NET MVC newsgroup about this recently. It's pretty straight forward, you simply use the standard HTML input tag (type="file") in your form and then iterate through the HttpRequest's Files collection in your controller.

Here's the view's HTML:

<form action="/Document.mvc/UpdateDocument/" method="post" enctype="multipart/form-data">
   <label for="file1">Document 1</label>
   <input type="file" id="file1" name="file1" />

   <label for="file2">Document 2</label>
   <input type="file" id="file2" name="file2" />

   <label for="file3">Document 3</label>
   <input type="file" id="file3" name="file3" />
</form>

And here's the controller action:

public void SaveDocuments()
{
   foreach (string inputTagName in Request.Files)
   {
       HttpPostedFile file = Request.Files[inputTagName];
       if (file.ContentLength > 0)
       {
           string filePath = Path.Combine(@"C:\MyUploadedFiles", Path.GetFileName(file.FileName));
           file.SaveAs(filePath);
       }
   }
}

Thursday, March 20, 2008

Using the IRepository pattern with LINQ to SQL

27th March 2009. It's now a year since I wrote this post. Thanks to some comments by Janus2007 I've realised that it needs updating. I've replaced the code with the current version of the Suteki Shop LINQ generic repository. There are a number of changes in the way it works. The most obvious, and one that I should have updated a long time ago is the GetAll method returns  an IQueryable<T> rather than an array. I actually changed this soon after I wrote the post, but totally forgot about the naive implementation given here.

The other major change is marking the SubmitChanges methods as obsolete. Jeremy Skinner, who has been doing some excellent work on Suteki Shop has pushed this change. UoW (DataContext) management is now handled by attributes on action methods.

Please have a look at the Suteki Shop code to see the generic repository in action:

LINQ to SQL is a quantum leap in productivity for most mainstream .NET developers. Some folks may have been using NHibernate or some other ORM tool for years, but my experience in a number of .NET shops has been that the majority of developers still hand code their data access layer. LINQ is going to bring some fundamental changes to the way we architect our applications. Especially being able to write query style syntax directly in C# against both a SQL Server database and against in memory object graphs begs some interesting questions about application architecture.

So where is our point of separation between data access and domain? Surely I'm not recommending that we abandon a layered architecture and write all our data access directly into our domain classes?

My current project is based on the new MVC Framework. I've been using LINQ to SQL for data access as well as an IoC container (Windsor) and NUnit plus Rhino Mocks for testing. For my data access layer I've used the IRepository pattern popularized by Ayende in his excellent MSDN article on IoC and DI. My Repository looks like this:

using System;
using System.Linq;
using System.Linq.Expressions;
using System.Data.Linq;
using Suteki.Common.Extensions;
namespace Suteki.Common.Repositories
{
    public interface IRepository<T> where T : class
    {
        T GetById(int id);
        IQueryable<T> GetAll();
        void InsertOnSubmit(T entity);
        void DeleteOnSubmit(T entity);
		[Obsolete("Units of Work should be managed externally to the Repository.")]
        void SubmitChanges();
    }
    public interface IRepository
    {
        object GetById(int id);
        IQueryable GetAll();
        void InsertOnSubmit(object entity);
        void DeleteOnSubmit(object entity);
		[Obsolete("Units of Work should be managed externally to the Repository.")]
        void SubmitChanges();
    }
    public class Repository<T> : IRepository<T>, IRepository where T : class
    {
        readonly DataContext dataContext;
        public Repository(IDataContextProvider dataContextProvider)
        {
            dataContext = dataContextProvider.DataContext;
        }
        public virtual T GetById(int id)
        {
            var itemParameter = Expression.Parameter(typeof(T), "item");
            var whereExpression = Expression.Lambda<Func<T, bool>>
                (
                Expression.Equal(
                    Expression.Property(
                        itemParameter,
                        typeof(T).GetPrimaryKey().Name
                        ),
                    Expression.Constant(id)
                    ),
                new[] { itemParameter }
                );
            return GetAll().Where(whereExpression).Single();
        }
        public virtual IQueryable<T> GetAll()
        {
            return dataContext.GetTable<T>();
        }
        public virtual void InsertOnSubmit(T entity)
        {
            GetTable().InsertOnSubmit(entity);
        }
        public virtual void DeleteOnSubmit(T entity)
        {
            GetTable().DeleteOnSubmit(entity);
        }
        public virtual void SubmitChanges()
        {
            dataContext.SubmitChanges();
        }
        public virtual ITable GetTable()
        {
            return dataContext.GetTable<T>();
        }
        IQueryable IRepository.GetAll()
        {
            return GetAll();
        }
        void IRepository.InsertOnSubmit(object entity)
        {
            InsertOnSubmit((T)entity);
        }
        void IRepository.DeleteOnSubmit(object entity)
        {
            DeleteOnSubmit((T)entity);
        }
        object IRepository.GetById(int id)
        {
            return GetById(id);
        }
    }
}

As you can see, this generic repository insulates the rest of the application from the LINQ to SQL DataContext and provides basic data access methods for any domain class. Here's an example of it being used in a simple controller.

using System.Web.Mvc;
using Suteki.Common.Binders;
using Suteki.Common.Filters;
using Suteki.Common.Repositories;
using Suteki.Common.Validation;
using Suteki.Shop.Filters;
using Suteki.Shop.Services;
using Suteki.Shop.ViewData;
using Suteki.Shop.Repositories;
using MvcContrib;
namespace Suteki.Shop.Controllers
{
	[AdministratorsOnly]
    public class UserController : ControllerBase
    {
        readonly IRepository<User> userRepository;
        readonly IRepository<Role> roleRepository;
    	private readonly IUserService userService;
    	public UserController(IRepository<User> userRepository, IRepository<Role> roleRepository, IUserService userService)
        {
            this.userRepository = userRepository;
            this.roleRepository = roleRepository;
        	this.userService = userService;
        }
        public ActionResult Index()
        {
            var users = userRepository.GetAll().Editable();
            return View("Index", ShopView.Data.WithUsers(users));
        }
        public ActionResult New()
        {
            return View("Edit", EditViewData.WithUser(Shop.User.DefaultUser));
        }
		[AcceptVerbs(HttpVerbs.Post), UnitOfWork]
		public ActionResult New(User user, string password)
		{
			if(! string.IsNullOrEmpty(password))
			{
				user.Password = userService.HashPassword(password);
			}
			try
			{
				user.Validate();
			}
			catch(ValidationException ex)
			{
				ex.CopyToModelState(ModelState, "user");
				return View("Edit", EditViewData.WithUser(user));
			}
			userRepository.InsertOnSubmit(user);
			Message = "User has been added.";
			return this.RedirectToAction(c => c.Index());
		}
        public ActionResult Edit(int id)
        {
            User user = userRepository.GetById(id);
            return View("Edit", EditViewData.WithUser(user));
        }
		[AcceptVerbs(HttpVerbs.Post), UnitOfWork]
		public ActionResult Edit([DataBind] User user, string password)
		{
			if(! string.IsNullOrEmpty(password))
			{
				user.Password = userService.HashPassword(password);
			}
			try
			{
				user.Validate();
			}
			catch (ValidationException validationException) 
			{
				validationException.CopyToModelState(ModelState, "user");
				return View("Edit", EditViewData.WithUser(user));
			}
			return View("Edit", EditViewData.WithUser(user).WithMessage("Changes have been saved")); 
		}
        public ShopViewData EditViewData
        {
            get
            {
                return ShopView.Data.WithRoles(roleRepository.GetAll());
            }
        }
    }
}

Because I'm using an IoC container I don't have to do any more than request an instance of IRepository<User> in the constructor and because the Windsor Container understands generics I only have a single configuration entry for all my generic repositories:

<?xml version="1.0"?>
<configuration>
  <!-- windsor configuration. 
  This is a web application, all components must have a lifesytle of 'transient' or 'preWebRequest' -->
  <components>
    <!-- repositories -->
    <!-- data context provider (this must have a lifestyle of 'perWebRequest' to allow the same data context
    to be used by all repositories) -->
    <component
      id="datacontextprovider"
      service="Suteki.Common.Repositories.IDataContextProvider, Suteki.Common"
      type="Suteki.Common.Repositories.DataContextProvider, Suteki.Common"
      lifestyle="perWebRequest"
     />
	<component
		id="menu.repository" 
		service="Suteki.Common.Repositories.IRepository`1[[Suteki.Shop.Menu, Suteki.Shop]], Suteki.Common"
		type="Suteki.Shop.Models.MenuRepository, Suteki.Shop"
		lifestyle="transient"
		/>
		
    <component
      id="generic.repository"
      service="Suteki.Common.Repositories.IRepository`1, Suteki.Common"
      type="Suteki.Common.Repositories.Repository`1, Suteki.Common"
      lifestyle="transient" />
    
	....
  </components>
  
</configuration>

The IoC Container also provides the DataContext. Note that the DataContext's lifestyle is perWebRequest. This means that a single DataContext is shared between all the repositories in a single request.

To test my UserController I can pass an object graph from a mock IRepository<User>.

[Test]
public void IndexShouldDisplayListOfUsers()
{
    User[] users = new User[] { };
    UserListViewData viewData = null;

    using (mocks.Record())
    {
        Expect.Call(userRepository.GetAll()).Return(users);

        userController.RenderView(null, null);
        LastCall.Callback(new Func<string, string, object, bool>((v, m, vd) => 
        {
            viewData = (UserListViewData)vd;
            return true;
        }));
    }

    using (mocks.Playback())
    {
        userController.Index();
        Assert.AreSame(users, viewData.Users);
    }
}

Established layered architecture patterns insulate the domain model (business objects) of a database from the data access code by layering that code into a data access layer that provides services for persisting and de-persisting objects from and to the database. This layered approach becomes essential as soon as you start doing Test Driven Development which requires you to test you code in isolation from your database.

So is LINQ data access code? I don't think so. Because the syntax for querying in-memory object graphs is identical to that for querying the database it makes sense to place LINQ queries in your domain layer. During testing, the component under test can work with in memory object graphs, but when integrated with the data access layer those same queries become SQL queries to a database.

Here's a simple example. I've got the canonical Customer->Orders->OrderLines business entities. Now say I get an Customer from my IRepository<Customer>, I can then query my Customer's Orders using LINQ inside my controller:

Customer customer = customerRepository.GetById(customerId);
int numberOfOrders = customer.Orders.Count(); // LINQ

I unit test this by returning a fully formed Customer with Orders from my mock customerRepository, but when I run this code against a concrete repository LINQ to SQL will construct a SQL statement, something like:

SELECT COUNT(*) FROM Order WHERE CustomerId = 34.