Fustrating Visual Studio Templates

VS 2005 has a new template feature that allows you to easily create item and project templates that you can add to the 'new item' and 'new project' dialogue boxes. It's pretty cool, you just select an item or a project from an existing solution and click file->Export template, a wizard pops up that allows you to name the template and optionally deploy it. Scott Guthrie has a nice post about it on his excellent blog (a required read for .net developers). As well as projects and items there's also a multi-project template that you're supposed to be able to use to produce template solutions. I've been playing with this for last couple of days in order to create a web service solution template for our team and it's pretty disapointing. Not much thought seems to have gone into it and there are two things in particular that make it useless for me. First of all, there's no way of customising the name of the projects in the solution. In the new solution dialogue you can choose the name of your solution and most .net projects I've worked on also use this as a solution namespace. Usually you name the projects in the solution prefixed with the solution name which plays well with the default namespace behaviour in visual studio. For example, say I call my solution 'Mike.MegaThing'. I'd then name my projects things like, 'Mike.MegaThing.Domain', 'Mike.MegaThing.Service', 'Mike.MegaThing.Dal'. Now every time I create a new class in 'Mike.MegaThing.Domain', the namespace is automatically created as 'Mike.MegaThing.Domain', perfect. Unfortunately the multi-project-template has no way of allowing you to change the name of it's projects. Although template items, like classes, can be tokenised and expanded with a range of default parameters like $safeprojectname$ this doesn't apply to the .vstemplate files themselves. My solution template has to have fixed project names like 'Service' or 'Domain' with similar default namespaces, which is crap because I want 'Mike.MegaThing.Dal' to be in a different namespace to 'Mike.LittleThing.Dal'. The other thing which makes the multi-project template useless is the wierd way it creates the folder structure for your created solution. Normally, if you add a project to a solution it creates a sub folder under your solution folder with the project name and puts the .csproj file in there. The multi-project template doesn't do this, instead it creates a sub folder under the solution folder with the name of the solution and then creates the project folders in that. There seems to be no way of altering this behaviour. So, thumbs down to visual studio templates for creating template solutions, although if you just want simple project or item templates it's perfectly adequate. I'm now going to be digging into the Guidance and Automation Toolkit (GAT) which looks much closer to what I need, more soon.

Software Factories, Domain Specific Languages and related stuff

I've recently had to get up to speed on some new buzzwords: 'Domain Specific Language' (DSL), 'Software Factories' and the 'Guidance and Automation Toolkit' (GAT). What do they all mean, and how do they relate to each other? I found it quite hard to find out. Two of them are Microsoft specific; 'Software Factories' and GAT, the other (DSL) is a generic software term, but is also being used for a specific Microsoft product. Martin Fowler explains it here extremely well. Before I get down to the terms themselves, first a little background... There's a constant trend in software development for common tasks to be automated. That's what computers are for. Of course the automation is a very tricky thing to get right and takes a while to evolve. First assemblers automated the conversion of human readable codes to ones and zeros, then compilers automated common tasks in a way that allowed code to be written at a higher level of abstraction. At the same time operating systems and now frameworks gradually handled more and more housekeeping tasks such as opening and closing files, network communication and printing stuff. There were a few false dawns, usually when the level of abstraction is raised too quickly without understanding all the implications, the case tools of the 80's and 90's are a good example. They never really succeeded because they couldn't bridge the gap between code generation and customisation. But now we're on the cusp of a new breakthrough in the level of abstraction we use to build common application types. The nirvana of the business developer is to be able to concentrate on capturing the business entities, relationships and rules and then to press a magic button that creates the application according to current best practices. We're getting closer and closer to this these days. The .net framework provides a huge level of abstraction for common tasks, the Enterprise Application Blocks make it even easier, and it's common to use either code generators such as code smith, or object/relational mappers such as NHibernate to automate data access. If you know all the tools out there and how to use them you can almost reach that nirvana. You can imagine a kind of 'automation of automation' where you might choose say an 'Enterprise smart client application' package that would know how to use all these frameworks and code generators to give you such a magic button. That in short is what Software Factories are. It's Microsoft's word for a collection of code generators and frameworks as well as a mechanism to capture the developer's intentions, that allows the automatic generation of all non business specific code. So the top level abstraction in this set of concepts is the Software Factory. At the practical, actually available today in beta level are the GAT and DSL Tools. I was quite confused about how these played together when I first started looking at them. They seem to do similar things, but which one to use when? In fact they are products of two separate teams at Microsoft who weren't really aware of what the other was doing until they'd progressed some way down their separate roads. The GAT was developed by the Patterns and Practices group; the people who make the Enterprise Application Blocks and produce advice on application architecture. It basically builds on Visual Studio Templates, adding a code templating system similar to code smith and some nice extension points that allow you to create wizards and actions. They've already released a number of software factories based on the GAT, such as the 'Service Factory'. The GAT is the easy-to-use, every software house should have one, way of building software factories. The DSL tools come as part of the Visual Studio SDK, produced by the Visual Studio team. The VS SDK is aimed at people who want to produce extensions for Visual Studio and is a bit more hard core than the GAT. I haven't looked at DSL tools in anywhere near the same depth, but I understand that it's essentially the diagramming engine used by the class designer and various other Visual Studio tools linked to a code generation engine. Apparently both the Visual Studio team and the Patterns and Practices team were working on their own code templating engines and it was only when someone noticed the duplicated effort that the two teams were brought together. The result was the T4 engine that used in both tools. These two toolsets represent quite an exciting development by Microsoft and as I said above, there's an expectation that we're on the cusp of a significant step change in software development. You only have to read about some of the huge productivity benefits of things like Ruby on Rails to appreciate how easy life can be with the right tools.

Reflecting Generics

I had an interesting problem today, how do you find out if an object is a subclass of a generic type? If you've simply got an instance of a generic type...

List<int> myobject = new List<int>();

You can find that myobject is a List<> by using the method GetGenericTypeDefinition():

if(myobject.GetType().GetGenericTypeDefinition() == typeof(List<>)) { ... }

But say you've got a type that inherits from a generic type:

public class A : List<int> { }

Now if you've got an instance of A:

A myA = new A();

You can't call GetGenericTypeDefinition(), it complains that myA.GetType() is in an incorrect state. Similarly, asking this:

myA.GetType().IsSubclassOf(typeof(List<>))

returns false because myA is a subclass of List<int>, but not List<>. I couldn't find any way of doing this without manually walking up the class hierarchy. In the end I gave in and wrote a this little recursive function:

public bool IsDerivedFromGenericType(Type givenType, Type genericType)
{
    Type baseType = givenType.BaseType;
    if (baseType == null) return false;
    if (baseType.IsGenericType)
    {
        if (baseType.GetGenericTypeDefinition() == genericType) return true;
    }
    return IsDerivedFromGenericType(baseType, genericType);
}

I guess where I was going wrong was thinking that List<int> 'inherits' from List<>, but of course it doesn't. It's an entirely unrelated hierarchy. Generics are not inheritance. I spent some time digging through the newsgroups looking for an answer to this and there's quite a lot of confusion around this issue. A really common misconception is to think that, for example, List<int> inherits from List<object> and that you can downcast like this..

List<int> myIntList = new List<int>();
List<object> myObjectList = (List<object>)myIntList;

It doesn't work.

A nice soap test tool: SoapUI

Do you manually code test harnesses for your web services? It's a real pain, and because every web service is defined by its wsdl you shouldn't have to do it. I've been using a soap test tool called SoapUI for the last few weeks. It makes experimenting with raw web service requests a breeze. You just have to give it a wsdl url and it will construct an example request which you can fill in with some test values. Then it's just a question of waiting for the response. It saves all the different requests you make in a project file and you can change the endpoint to try out different deployments. It's also got facilities to do load testing and generate statistics for web services, but I haven't had a need to use that yet. The only downside about this tool is that it's got a nasty javaesque interface, cross platform UIs always suck, but this one is not too bad.

Functional Programming

I've just read a really good blog on functional programming by defmacro.org (I couldn't find his/her name on the blog anywhere). I didn't know that Alan Turing, Kurt Godel and John von Neumann all worked together at Princeton. I guess it makes sense that those three guys would know each other, but just imagine being able to listen in to their conversations. Hmm, probably completely over my head:) Anyway, there was a fourth guy, who I'd not heard of before, Alonzo Church. He invented an alternative to the Turing machine called lambda calculus. Of course the Turing machine went on to be the basis of pretty much all modern computers, but Alonzo wasn't entirely forgotten. In the late 50's LISP was invented as a programming language that implemented lambda calculus and is still with us. It's never really been mainstream, but the Emacs text editor is written in LISP, for example. One think that I've read in several places about LISP is that it makes some things, like creating domain specific languages much easier. So why is functional programming interesting? And what can you do that you can't do with imperative programming languages like the C* family? Well, it all goes back to the lambda calculus. I don't want to repeat defmacro's excellent article, but you can do all kinds of cool stuff, like mathematically redictable unit testing, simple parallel processing with no chance of race conditions, and even hot deployment where you can drop in code changes to a running program. Also because of the nature of functional programs (they are, in effect, a single function) you can use the lambda calculus to do automated optimisations to your program or do lazy execution. Also, because a functional language's state is always represented by its stack, unlike an imperative language where the state is a combination of the stack and the heap, you can use 'continuations' to halt and restart your program, or even pass your program's state to a separate program (it's also what allows for hot deployment). Now that C# is begining to get some funcional programming constructs (lambda expressions, expression trees, delayed execution), functional programming is going to become a mainstream concern for dot net developers. Joel Spolsky, has a nice post, Can your programming language do this? It shows some stuff that becomes much easier once functions become first class citizens, and there's a post about it on the powershell blog, showing some of the powerfull functional features of that language. If you look at everyone's current favorite language, Ruby, it already does all this stuff, as well as providing a full compliment of object oriented features. Unfortunately, some of the really cool stuff I mentioned above, like parallelisation or automated optimisations, can only be done with 'pure' functional language like Haskell or Erlang. It's making me think that I'd like to dig into a good Haskell book sometime.

Writing web services contract first using WSCF

Web service gurus will always tell you to write web services "contract first". This means defining the schema for your service types, your messages and wsdl definitions before creating your implementation, which is the exact opposite of what you do when you work with Microsoft's tools. The visual studio way of doing it treats the wsdl as a behind-the-scenes artifact that the developer shouldn't need to worry his little head about. If you're using the Visual Studio tool set end-to-end you don't need to be bothered with wsdl or any of the other nasty angle bracket stuff to do with web services. The problem comes when you need to support interoperability with non dot net applications, then attributing your dot net types to produce the correct wsdl schema can be quite tricky. It would be much nicer if you could write your schemas, message definitions and wsdl first and then generate most of the boiler place asmx code from the wsdl. I was recently introduced, by the software architect where I'm contracting at the moment (hello Shelly), to WSCF (Web Service Contract First) which is a free tool that does just that. It's a free download from thinktecture, they're the consultancy headed by Ingo Rammer the remoting guy. It works as a visual studio plug in, all you have to do is write your xsds that define your types and messages (I use xml spy for doing that) and then you just right click on the message xsd file in the solution explorer, and choose 'Create WSDL Interface Description', a nice wizard pops up where you choose some defauls and then whoosh, there's your wsdl. After you've got the wsdl file you can right click on it and choose 'Generate Web Service Code', again a nice wizard pops and before you know it, you've got all your service types and framework generated in C#. Very nice. Thinktecture have also gone to the effort of writing a really nice walk-through for WSCF that gets you up and running really fast, it only took me a couple of hours to produce my first web service with this method yesterday.

Playing with HTTP.SYS

I've been having fun playing with HTTP.SYS and the HttpListener (NET 2.0) class in System.Net. HTTP.SYS is the new kernel mode http listener that replaces winsock in 2003 Server and XP SP2 and the HttpListener class is a nice managed interface for the HTTP.SYS API. It's really easy to write your own web server and do cool things like URL mangling. Basically you just create a new instance of HttpListener and then tell it to hand you any requests that match the urls you give it. It's easy to imgaine writing a RESTfull service (see my previous post on this) using HttpListener. You could just write a windows service to receive any requests, say 'http://mikehadlow.com/restapp/' and then parse the local address part of the url to tell you which object to get from your database. So, say we got the url 'http://mikehadlow.com/restapp/customer/1234', we'd parse the local address 'restapp/customer/1234', grap 'customer' and '1234' and go to our database and 'select * from customer where id = 1234', serialise our customer object as xml and return that xml in the response. You can also host the ASP.NET pipeline in your own application using HttpListener. There's a really good MSDN article on this here. This means you no longer have to worry about having IIS installed on a machine to make your application response to http requests, any app can do it including windows forms apps, console apps and windows services. Here's some code for a little console app, showing how you use HttpListener. It's just a slight re-working of the example code from the help file. Just start up the app and browse to 'http://localhost:1234/something/else/here?somekey=somevalue&someotherkey=someothervalue' and you'll see it echo back the local address and the query string parameters.

using System;
using System.Net;

namespace HttpListenerTest
{
    class Program
    {
        HttpListener _listener = new HttpListener();

        static void Main(string[] args)
        {
            Program program = new Program();
            program.Start();
        }

        public void Start()
        {
            _listener.Prefixes.Add("http://*:1234/");
            _listener.Start();
            Console.WriteLine("Listening, hit enter to stop");
            _listener.BeginGetContext(new AsyncCallback(GetContextCallback), null);
            Console.ReadLine();
            _listener.Stop();
        }

        public void GetContextCallback(IAsyncResult result)
        {
            HttpListenerContext context = _listener.EndGetContext(result);
            HttpListenerRequest request = context.Request;
            HttpListenerResponse response = context.Response;

            System.Text.StringBuilder sb = new System.Text.StringBuilder();

            sb.Append("");
            sb.Append(string.Format("HttpMethod: {0}
", request.HttpMethod));
            sb.Append(string.Format("Uri:        {0}
", request.Url.AbsoluteUri));
            sb.Append(string.Format("LocalPath:  {0}
", request.Url.LocalPath));
            foreach (string key in request.QueryString.Keys)
            {
                sb.Append(string.Format("Query:      {0} = {1}
", key, request.QueryString[key]));
            }
            sb.Append("");

            string responseString = sb.ToString();
            byte[] buffer = System.Text.Encoding.UTF8.GetBytes(responseString);
            response.ContentLength64 = buffer.Length;

            using (System.IO.Stream outputStream = response.OutputStream)
            {
                outputStream.Write(buffer, 0, buffer.Length);
            }
            _listener.BeginGetContext(new AsyncCallback(GetContextCallback), null);
        }
    }
}

Resting?

I've just become aware of a new term: REST. It stands for Representational State Transfer. It's kinda like a 'back to basics' movement about web standards and how the internet is developing. As time goes on, protocols have been layered on top of protocols. In the begining was IP, then HTTP got layered on top, subsequently SOAP was layered on top of HTTP and now we have a whole load of WS-* protocols being layered on top of SOAP. The REST people are basically saying that a lot of the new protocols are duplicating stuff that the older lower level protocols do already. Why have a SOAP action when HTTP already gives you CRUD verbs; GET, POST, PUT, DELETE? Why do we need to have identifiers when we already have the URL? A RESTafarian (I didn't make that up) would say you can do web services without SOAP or any of that WS-* stuff. Say I want to get a customer. All I'd need to do is carry out an HTTP GET for this URL: http://www.thecompany.com/enterprise/customer/0123456 That would return me a POX (Plain Old Xml, didn't make that up either) representation of my customer. RESTafarians would also argue that my customer resource should only have stuff about the customer itself, related resources (orders, invoices) should be represented by links to those resources. If it all seems slightly familiar, it is, of course, just how the simple HTML web works and the simplicity of the model is the World Wide web's fundamental strength. It's a persuasive argument and a real challenge to the current dominance (at least in the Microsoft world that I live in) of SOAP RPC style web services. Now, can REST be done with WCF, that's a question!

one line fibonacci

I've been playing with windows powershell (aka monad, aka MSH) recently. It's got some really great functional programming constructs. Here's a one line fibonacci...

PS C:\> 1..10 | foreach{ $i = $j = 1 }{ $k = $i+$j; $i=$j; $j=$k; $k }
2
3
5
8
13
21
34
55
89
144

"1..10" generates a range of numbers from one to ten "|" is a pipe to the next command, "foreach" applies the first statement block once and then the second statement block for each item generated from the previous list.

The Vietnam of Computer Science

Ted Neward has written a great post on one of my favorite subjects: the object relational problem. He calls it 'The Vietnam of Computer Science'. I've thought for a long time that the OR mapping problem is currently the main challenge in developing business applications. I've faced these issues head on when I developed my own data-access-layer code-generator. Funnily enough, although it's such a huge issue in application design, there aren't any really good books on the subject (that I know of). Martin Fowler, Eric Evans and Rocky Lhotka all touch on the issues in their books on enterprise application development, but I've never seen it exhaustively covered anywhere. Ted's post is an excellent summary of the main compromises you have to make, but I'd really like someone to write the definitive book on the subject.