Wednesday, February 27, 2008

Nasty IE margin inheritance bug

I've recently been caught out by a nasty IE bug. The symptom is form elements that have strangely large left margins that are not defined in your style sheet:

ie-margin-bug

Luckily it's quite a well known bug and once I stumbled upon the correct keywords to google (margin inheritance bug) there's plenty of advice out there on how to fix it. I found this blog post very useful, it's a shame the guy's stopped blogging there's some great stuff there; I love this pie chart. For me, it was a simple case of getting rid of my form div's margin-left property. Since my left menu floats left anyway the form will sit nicely to the right without the margin. The form elements inherit the margin of their container, so getting rid of it removes the problem.

Monday, February 11, 2008

Extension method validators

I'm building an MVC Framework application at the moment and I wanted a simple, easy to grep way of validating fields passed back from forms. In MVC-land all the form variables are passed as parameters to the action method. Extension methods turned out to be a very neat solution for doing validation and data conversion. Here's an example:

[ControllerAction]
public void UpdateContact(
    int contactId,
    string name,
    string address1,
    string address2,
    string address3,
    string county,
    string postcode,
    string telephone,
    string email)
{
    try
    {
        name.Label("Name").IsRequired();
        address1.Label("Address 1").IsRequired();
        county.Label("County").IsRequired();
        postcode.Label("Postcode").IsRequired().IsPostcode();
        telephone.Label("Telephone").IsRequired().IsTelephoneNumber();
        email.Label("Email").IsRequired().IsEmail();
    }
    catch (ValidationException exception)
    {
        ContactViewData viewData = ContactViewData(contactId);
        viewData.ErrorMessage = exception.Message;
        RenderView("Contact", viewData);
        return;
    }

 // update the contact and render the view
}

As you can see we pass the contact's details from a form. In the try block the extension method validators are called. Any of them can raise a validation exception which is caught by the catch block and a view is rendered showing the validation error.

The 'Label' extension returns a ValidateField instance that can be consumed by any other validator, this is so that we can raise exceptions that can be displayed directly to the user:

public static ValidateField Label(this string value, string label)
{
    return new ValidateField { Value = value, Label = label };
}

The 'IsRequired' extension takes a ValidateField and checks that the value is not null or empty:

public static ValidateField IsRequired(this ValidateField validateField)
{
    if (string.IsNullOrEmpty(validateField.Value))
    {
        throw new ValidationException(string.Format("{0} is required", validateField.Label));
    }
    return validateField;
}

And finally the 'IsEmail' extension uses a Regex to validate the string value:

public static ValidateField IsEmail(this ValidateField validateField)
{
    // ignore is null or empty, use IsRequired in parrallel to check this if needed
    if (string.IsNullOrEmpty(validateField.Value)) return validateField;

    string patternLenient = @"\w+([-+.]\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*";
    if (!Regex.Match(validateField.Value, patternLenient).Success)
    {
        throw new ValidationException(string.Format("{0} must be a valid email address", validateField.Label));
    }
    return validateField;
}

I'm finding extension methods a real boon for writing DSL-ish APIs. I'll leave the 'IsTelephone' and 'IsPostcode' exercises for your enjoyment :)

Wednesday, February 06, 2008

RESTful file uploads with HttpWebRequest and IHttpHandler

I've currently got a requirement to transfer files over a web service. I would normally have used MTOM, but after reading Richardson & Ruby's excellent RESTful Web Services and especially their description of the Amazon S3 service, I decided to see how easy it would be to write a simple file upload service using a custom HttpHandler on the server and a raw HttpWebRequest on the client. It turned out to be extremely simple.

First implement your custom HttpHandler:

public class FileUploadHandler : IHttpHandler
{
    const string documentDirectory = @"C:\UploadedDocuments";

    public bool IsReusable
    {
        get { return false; }
    }

    public void ProcessRequest(HttpContext context)
    {
        string filePath = Path.Combine(documentDirectory, "UploadedFile.pdf");
        SaveRequestBodyAsFile(context.Request, filePath);
        context.Response.Write("Document uploaded!");
    }

    private static void SaveRequestBodyAsFile(HttpRequest request, string filePath)
    {
        using (FileStream fileStream = File.Open(filePath, FileMode.Create, FileAccess.Write))
        using (Stream requestStream = request.InputStream)
        {
            int bufferSize = 1024;
            byte[] buffer = new byte[bufferSize];
            int byteCount = 0;
            while ((byteCount = requestStream.Read(buffer, 0, bufferSize)) > 0)
            {
                fileStream.Write(buffer, 0, byteCount);
            }
        }
    }
}

As you can see, it's simply a question of implementing IHttpHandler. The guts of the operation is in the ProcessRequest message, and all we do is save the input stream straight to disk. The SaveRequestBodyAsFile method just does a standard stream-to-stream read/write.

To get your handler to actual handle a request, you have to configure it in the handers section of the Web.config file, for examle:

<httpHandlers>
 <add verb="*" path="DocumentUploadService.upl" validate="false" type="TestUploadService.FileUploadHandler, TestUploadService"/>
</httpHandlers>

Here I've configured every request for 'DocumentUploadService.upl' to be handled by the FileUploadHandler. There are a couple of more things to do, first, if you don't want to configure IIS to ignore requests for non-existent resources you can put an empty file called 'DocumentUploadService.upl' in the root of your web site. You also need to configure IIS so that requests for .upl files (or whatever extension you choose) are routed to the ASP.NET engine. I usually just copy the settings for .aspx files.

On the client side you can execute a raw HTTP request by using the HttpWebRequest class. Here's my client code:

public void DocumentUploadSpike()
{
    string filePath = @"C:\Users\mike\Documents\somebig.pdf";

    string url = "http://localhost:51249/DocumentUploadService.upl";
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.Accept = "text/xml";
    request.Method = "PUT";

    using(FileStream fileStream = File.OpenRead(filePath))
    using (Stream requestStream = request.GetRequestStream())
    {
        int bufferSize = 1024;
        byte[] buffer = new byte[bufferSize];
        int byteCount = 0;
        while ((byteCount = fileStream.Read(buffer, 0, bufferSize)) > 0)
        {
            requestStream.Write(buffer, 0, byteCount);
        }
    }

    string result;

    using (WebResponse response = request.GetResponse())
    using (StreamReader reader = new StreamReader(response.GetResponseStream()))
    {
        result = reader.ReadToEnd();
    }

    Console.WriteLine(result);
}

Here too we simply stream a file straight into the request stream and then call GetResponse on the HttpWebRequest. The last bit just writes the response text to the console. Note that I'm using the HTTP method PUT rather than POST, that's because we're effectively adding a resource. The resource location I'm adding should be part of the URL. For example, rather than:

http://localhost:51249/DocumentUploadService.upl

it should really look more like:

http://localhost:51249/mikehadlow/documents/2345

Indicating that user mikehadlow is saving a file to location 2345. To make it work would simply be a case of implementing some kind of routing (the MVC Framework would be ideal for this).

Sunday, February 03, 2008

Never write a for loop again! Fun with Linq style extension methods.

One of the things I like about Ruby is the range operator. In most C style languages in order to create a list of  numbers you would usually use a for loop like this:

List<int> numbers = new List<int>();
for (int i = 0; i < 10; i++)
{
    numbers.Add(i);
}
int[] myArray = numbers.ToArray();

But in Ruby you just write this:

myArray = o..9.to_a

But now with extension methods and custom iterators we can do the same thing in C#. Here's a little extension method 'To':

public static IEnumerable<int> To(this int initialValue, int maxValue)
{
    for (int i = initialValue; i <= maxValue; i++)
    {
        yield return i;
    }
}

You can use it like this:

1.To(10).WriteAll();

1 2 3 4 5 6 7 8 9 10

Note the WriteAll() method, that's an extension method too, it simply writes each item in the list to the console:

public static void WriteAll<T>(this IEnumerable<T> values)
{
    foreach (T value in values)
    {
        Console.Write("{0} ", value);
    }
    Console.WriteLine();
}

You can mix your custom extension methods with the built in Linq methods, let's count up to fifty in steps of ten:

1.To(5).Select(i => i * 10).WriteAll();

10 20 30 40 50

Or maybe just output some even numbers:

1.To(20).Where(i => i % 2 == 0).WriteAll();

2 4 6 8 10 12 14 16 18 20

Here's another extension method 'Each', it just applies a Lambda expression to each value:

public delegate void Func<T>(T value);

public static void Each<T>(this IEnumerable<T> values, Func<T> function)
{
    foreach (T value in values)
    {
        function(value);
    }
}

Let's use it to output a ten by ten square of zero to ninety nine:

0.To(9).Each(i => 0.To(9).Select(j => (i * 10) + j).WriteAll());

0 1 2 3 4 5 6 7 8 9 
10 11 12 13 14 15 16 17 18 19 
20 21 22 23 24 25 26 27 28 29 
30 31 32 33 34 35 36 37 38 39 
40 41 42 43 44 45 46 47 48 49 
50 51 52 53 54 55 56 57 58 59 
60 61 62 63 64 65 66 67 68 69 
70 71 72 73 74 75 76 77 78 79 
80 81 82 83 84 85 86 87 88 89 
90 91 92 93 94 95 96 97 98 99 

Now here's something really cool, and more than a little dangerous. Because chaining extension methods of IEnumerable<T> means that you're effectively building a decorator chain of enumerators, you don't actually execute every iteration unless you ask for it. This means we can write infinite loop generators and then bound them by only asking for some members. Best to demonstrate. Here's a method that returns an infinite number of integers (not really, it'll end with an exception at int.MaxValue):

public static IEnumerable<int> Integers
{
    get
    {
        int i = 0;
        while (true)
        {
            yield return i++;
        }
    }
}

We can use it with the built in Linq method 'Take'. For example here we are printing out zero to nine:

Numbers.Integers.Take(10).WriteAll();

0 1 2 3 4 5 6 7 8 9

And here is Five to Fifteen:

Numbers.Integers.Skip(5).Take(11).WriteAll();

5 6 7 8 9 10 11 12 13 14 15

Why is it dangerous? If you put any operation in the chain that simply iterates all the values you'll get an infinite loop. So you couldn't say:

Numbers.Integers.Reverse.Take(10).WriteAll();

Let's wrap up with an infinite Fibonacci series:

public static IEnumerable<int> Fibonacci
{
    get
    {
        int a = 0;
        int b = 1;
        int t = 0;

        yield return a;
        yield return b;

        while (true)
        {
            yield return t = a + b;
            a = b;
            b = t;
        }
    }
}

And use it to print out the first ten Fibonacci numbers:

Numbers.Fibonacci.Take(10).WriteAll();

0 1 1 2 3 5 8 13 21 34

I'm really enjoying learning the possibilities of Linq style extension methods. It makes C# feel much more like a functional language and let's you write in a nicer declarative style. If I've tickled your interest, check out Wes Dyer's blog. Here he writes an ASCII art program using Linq. Nice!

Saturday, February 02, 2008

ALT.NET UK

02022008625

I just got back to Brighton after spending last night and all of today at the ALT.NET UK open spaces conference. It was inspired by the US ALT.NET conference that took place last summer and ably organized by Ian Cooper, Alan Dean and Ben Hall.  I was more than slightly suspicious about the idea of a load of programming geeks turning up with no agenda and just seeing what happened. I did come away thinking that this kind of event is no substitute for something with prepared speakers, but it was great; I really enjoyed myself. Without dressing it up in 'open spaces' language, it was a fantastic opportunity to get together with like minded uber-keen developers and chew the cud.

The night before we all put our suggestions for topics on the white board wall (see the photo above) and then retired to the pub. Ian, Alan and Ben organized them into topics the next morning. To kick off the day in one room, I'd suggested the topic of IoC containers after my talk at DDD. I briefly introduced the subject by describing how I'd come to the IoC game via TTD and Dependency Injection. Uber blogger Roy Osherove (who'd come all the way from Israel) then started up a really interesting discussion around the limits of IoC, or rather how doing TDD forces you to do DI rather than giving you a choice. It was a very good point, and one that hadn't really struck me before. Of course Roy works for Typemock, so he obviously interested in showing how Typemock can alleviate TDD's arm lock on your architecture. Personally I'm still in my honeymoon period with IoC containers and TDD and haven't found the edge cases yet where I feel like enforced DI is dragging my architecture or productivity down enough that I need to do something about it. The discussion continued about the depth of unit testing that's appropriate, how to test legacy code and tradeoff between integration and unit tests. All good stuff.

I hung out in the room discussing F# and all things functional for the rest of the morning and learnt a lot about the why-of-functional that I hadn't really appreciated before. I must fire up the F# shell again and have another play.  During the afternoon I failed to attend any sessions at all. Firstly I got into a very long and interesting conversation with Michael Foord. I've been reading his blog for a while and I caught his talk at Mix UK last summer, so it was good to get to chat to the man in person. He showed me Resolver, the python code generator / spreadsheet that his company is building. It's a very impressive piece of work that brings together the immediate graphical data manipulation of a traditional spreadsheet and any managed code that you want, all glued together with python. You can imagine building component-oriented financial software and binding it together under the spreadsheet front end. Or, building a model with the spreadsheet and simply taking the python code it generates, sticking it an assembly and then harnessing that model from your application. We also chatted about his upcoming book, Iron Python in Action, which I'll be getting a copy of as soon as it hits the (virtual) shelves.

For the rest of the afternoon I just stayed in the lobby area chatting. That was probably the best thing about the event for me, just being able to meet other .net geeks and talk about coding all day. Usually when I do that, even with many people who's full time job is coding, I tend to see eyes glazing over, but not today :) Hey, I'm already looking forward to the next one.

There's a wiki: http://altnetpedia.com/ that's going to act as a record of the discussions today. Hopefully, along with the mailing list, it can act as the nucleus of a growing UK ALT.NET community.

Thursday, January 24, 2008

Writing unit tests for an MVC Framework controller sucks

Over the Christmas holidays I had a lot of fun writing a web application using the Castle Project's Monorail. One of the joys of doing Monorail development is the outstanding support for unit testing that it provides. That's not really surprising when you read some of the Castle Project guys' blogs, they are all thoroughly Test Infected, and indeed one of the main driving forces behind adopting any framework that allows a separation of concerns is the ease which such a framework can support Test Driven Development.

During the last couple of weeks I've been embarking on a new commercial project. I've decided to use the new Microsoft MVC Framework simply because it's much easier to justify using a tool when Microsoft are behind it. And looking to the future it's highly likely that most professional MS shops will have MVC Framework skills to support such an application whereas Monorail skills are inevitably always going to be niche.

Monorail and the MVC Framework are quite similar, and having come from a Monorail project, I've found it pretty easy getting up to speed with the Microsoft offering. Some things are nicer: there's the typically good tool support, especially intellisense in the views and the strongly typed view data is an improvement over NVelocity (I haven't tried Brail). The routing engine is very nicely thought out and much better than the afterthought bolted onto Monorail.

The one thing that's really surprised me though is the lack of thought which has gone into support for unit testing controllers. Monorail supplies a BaseControllerTest class that you can inherit from for your controller tests that allows you to mock or access all the controller infrastructure. For example, say I want to test that my controller calls Response.Redirect passing a specific URL.

  • Make my controller test inherit from BaseControllerTest
  • Override the virtual method BuildResponse and provide a mock response
  • Pass my controller to the PrepareController method in BaseControllerTest. This sets up a mock infrastructure for the controller and at some point will call my overriden BuildResponse method.
  • In the test itself simply set an expectation for Response.Redirect to be called with the expected URL.
[TestFixture]
public class PosterLoginControllerTests : BaseControllerTest
{
    MockRepository mocks;
    PosterLoginController posterLoginController;

....

    protected override IMockResponse BuildResponse()
    {
        return mocks.CreateMock<IMockResponse>();
    }

    [SetUp]
    public void SetUp()
    {
  ....
  
        PrepareController(posterLoginController);
    }

    [Test]
    public void IndexShouldRedirectLoggedInPosterToAddJobTest()
    {
        string expectedRedirectUrl = "/Job/JobForm.rails";

        using (mocks.Record())
        {
   ....
   
            Response.Redirect(expectedRedirectUrl);
        }

        using (mocks.Playback())
        {
            posterLoginController.Index();
        }
    }

Now you can't do anything like this with the MVC framework. There's no BaseControllerTest and it's impossible to provide mocks for all the hooks in the Controller class. It's not completely decoupled from the infrastructure. The only way to successfully test a controller is to write a test subclass that inherits from the controller under test and overrides all the important methods that deal with the infrastructure. Because you have to write a new one for each controller it's a real overhead with lots of duplicated effort. Here's an example that I've had to write for my current project:

public class TestUserController : UserController
{
    string actualViewName;

    public string ActualViewName
    {
        get { return actualViewName; }
    }

    object actualViewData;

    public object ActualViewData
    {
        get { return actualViewData; }
    }

    Dictionary<string, object> redirectValues = new Dictionary<string, object>();

    public Dictionary<string, object> RedirectValues
    {
        get { return redirectValues; }
    }

    public TestUserController(
        IUserRepository userRepository,
        IRepository<Role> roleRepository,
        IRepository<InstructionType> instructionTypeRepository,
        IAuthorisationBuilder authorisationBuilder)
        : base(
        userRepository,
        roleRepository,
        instructionTypeRepository,
        authorisationBuilder)
    {

    }

    protected override void RenderView(string viewName, string masterName, object viewData)
    {
        this.actualViewName = viewName;
        this.actualViewData = viewData;
    }

    protected override void RedirectToAction(object values)
    {
        PropertyInfo[] properties = values.GetType().GetProperties();
        foreach (PropertyInfo property in properties)
        {
            redirectValues.Add(property.Name, property.GetValue(values, null));
        }
    }

    NameValueCollection form = new NameValueCollection();

    public override NameValueCollection Form
    {
        get
        {
            return form;
        }
    }
}

As you can see it's quite extensive and not something that I relish repeating for each controller. Phil Haack describes injecting mocks into the controller, but then in an update to the post says that it's something that now broken for the CTP and basically admits that the only way of doing it is to write test specific subclasses.

The more I do IoC and dependency injection (which works well with the MVC framework BTW) the more I wish framework builders did the same. It would make all these issues go away. I guess it's a request too far as long as IoC Containers remain a niche thing; supporting them is one thing, requiring them quite another.

Update

John Rayner from Conchango recently pointed out a post on Ayende's blog, showing how you can mock controllers and use extension methods to provide mocks of protected methods on the controller. It's a neat trick and gets over the need to build test sub-classes of your controller. But, it still doesn't remove the main point which is: we shouldn't have to do these crazy tricks; the controller should be testable.

Friday, January 11, 2008

DDD6 Feedback

I got my feedback from my DDD6 talk today. Ben's already blogged about his, so I feel honor bound to do the same. Me a copycat blogger? Surely not :-)

I was very pleased with it; most people gave me 4 or 5 (unless that's out of 20?) Some of the comments were very kind:

"best session by miles"

"This one was great. I've already started looking at Spring.NET and Windsor."

"Really enjoyed this session. I got the impression that Mike was a 'doer' rather than an academic."

Some of the other comments pointed out failings in my presentation that I need to work on if I'm going to do more in the future:

"A great topic because it was more about a concept rather than being "here is a product and this is how you might use it". There was however, too much jumping about between PP and code giving me the impression that maybe the session wasn't rehearsed."

"The presenter would have benefitted from using two screens. One to show the powerpoint slides and one to show the code demo."

"Buy a proper laptop :-)"

I definitely agree about 'jumping around between PP and code'. I think I must have looked like a yoyo because of my compulsion to stand up when addressing the slides and the need to sit down when showing the code. As for the laptop, I now have a VGA cable.

For some people I just didn't present it at the correct level:

"I am completely new to the subject and as such did not find this enough of an introduction."

I think this is the trickiest thing when doing technical presentations; do you assume your audience know nothing at all about the subject, or do you assume they are experts? When I've done workshops and presentations for my clients, I usually have a good idea of the competence of the audience, but at DDD, you just have to take a guess. In fact it's worse than that because you can guarantee there will be both complete newbies and battle hardened experts on your chosen topic. With the IoC container talk I tried to aim it at myself about two years ago; someone who's tried TDD, knows a bit about patterns, but hadn't really grepped the full power of IoC and hadn't yet encountered IoC containers.

All in all, it was a very satisfying day and a great opportunity to share ideas. I think it's wonderful that there's a thing like DDD where total nobodies like me can get a chance to rant about their favorite obsessions. Long may it continue.

You have to stay up late for ALT.NET

I was very excited to hear that we would be having our very own ALT.NET conference here in the UK. I waited with mounting anticipation to spend all day talking about ReSharper. But I was thwarted; unfortunately they could only book a room for ten people. It was full 0.05 seconds after the registration opened. I waited all night with my finger poised above my mouse, ready to click, but in a moment of weakness I had to answer a call of nature. By the time I got back Ben had made his announcement and the room was full.

Update

Sometimes it pays to write sarcastic blog posts. Alan Dean, one of the organisers of ALT.NET  very kindly got in touch and managed to squeeze me in. Thanks Alan!

Thursday, January 10, 2008

Rob Conery on choosing your data access method

A while back I asked 'Why are you still hand coding your data access layer?' I briefly covered a few of the choices available for automating data access, but Rob Conery does a much better job in this blog post: ASP.NET MVC: Choosing Your Data Access Method. He weighs up Linq-to-SQL, Subsonic (his own creation) and NHibernate. I've gone with NHibernate via the Castle project's Active Record for Jobvertizer, but there's a good chance that I'll be using Linq-to-SQL for my next commercial project. It'll be interesting to note the differences.

More on source repository hosting: What about Continuous Integration?

My previous post 'Source repository hosting: Why do it yourself?' talked about the possibility of using an off-site source repository for hosting commercial code. I got a great comment from Dave Verwer:

"This is exactly the decision I made when I started my company about 18 months ago and I have never looked back. I use CVSDude (who also do SVN, of course) and their service has been outstanding while I have been with them."

Well, if it's good enough for Dave... But how do you do continuous integration when your source control is off site? Continuous integration tools like Cruise Control rely on knowing when a source control update has taken place in order to kick-off a build. Apparently there is a way; here's an email reply I got back from Mark Bathie at CVSDude:

"Your can use our post-commit call back facility to call a URL on your
server, which passes variables relating to the last checkin (variables
detailed in our specification). Your CGI script will these variables and
perform whatever tasks are required i.e. updating Cruise Control, etc."

I'm keen to try it out now. I'm kicking off a new project in the next few days, so CVSDude say hello to your new customer :)