one line fibonacci

I've been playing with windows powershell (aka monad, aka MSH) recently. It's got some really great functional programming constructs. Here's a one line fibonacci...

PS C:\> 1..10 | foreach{ $i = $j = 1 }{ $k = $i+$j; $i=$j; $j=$k; $k }
2
3
5
8
13
21
34
55
89
144

"1..10" generates a range of numbers from one to ten "|" is a pipe to the next command, "foreach" applies the first statement block once and then the second statement block for each item generated from the previous list.

The Vietnam of Computer Science

Ted Neward has written a great post on one of my favorite subjects: the object relational problem. He calls it 'The Vietnam of Computer Science'. I've thought for a long time that the OR mapping problem is currently the main challenge in developing business applications. I've faced these issues head on when I developed my own data-access-layer code-generator. Funnily enough, although it's such a huge issue in application design, there aren't any really good books on the subject (that I know of). Martin Fowler, Eric Evans and Rocky Lhotka all touch on the issues in their books on enterprise application development, but I've never seen it exhaustively covered anywhere. Ted's post is an excellent summary of the main compromises you have to make, but I'd really like someone to write the definitive book on the subject.

Book Review: Framework Design Guidelines by Krzysztof Cwalina and Brad Abrams.

I've just finished reading Framework Design Guidelines. This book evolved from the design guidelines put together during the development of the .net framework. Both of the authors are program managers on the CLR and .net teams, so you get a great inside look at the thinking behind design decisions made during the development of the base class library. The book is peppered with little side bars where various Microsoft developers comment on the guidelines. These really help to bring the whole thing to life and you get an insight into the arguments that must have gone on over the pros and cons of different approaches. Like they say, designing a good reusable framework is hard and Microsoft have learnt from their past mistakes (did someone say MFC?). I really liked the initial discussion on general framework design where the authors describe how the .net team tried to juggle some of the conflicting requirements that any framework faces; how to keep it simple while providing a powerful tool; how to keep it as open as possible for future enhancements and how to keep it consistent. Having some of the design decisions around some of the best known components laid out in front of you really helps when it comes to understanding and using them. One thing I hadn’t really appreciated, but is obvious when it’s pointed out, is how the framework tries to juggle simplicity with power by using aggregate components for the common simple tasks that offer a façade over a more complex factored API. The book uses the System.IO namespace as an example where the File class provides the high level, easy to use abstraction while the stream classes provide the powerful API needed for more demanding tasks. Of course knowing this is great when you're using .net every day but it's also a real education in how to design good frameworks. Now, you might say, I'm an application developer, not a framework developer so why would I need to know this? But, as Martin Fowler says, following good OO design principles inevitably means that you are constantly designing frameworks, even if you're your only customer. Having said that, one of the best points for me in the book was the discussion on when you shouldn't stick strictly to OO principles, especially when designing components for visual tools to use. Although the book doesn’t once mention test driven development, it did reinforce my love of this way of developing by showing how you should always design from usage scenarios. Always write down how your API is going to be used first. I skimmed a lot of the stuff on naming conventions. If like me you’ve worked in several different enterprises on several different large systems, you’ll have probably discussed naming conventions until you’ve gone slightly silly in the head. But you’ve got to have them and I guess having them here in this book does give you something to point at the next time someone who’s been hiding in a hole for ten years suggests Hungarian. Chapter 7 on exceptions should be required reading for any .net developer. It's the best explanation of why you should use structured exception handling I've yet seen. I wish I'd had this book handy a couple of years ago when I spent hours arguing with an old school developer about the problems with error code return values. Chapter 9 on common design patterns should also be essential reading especially the discussion on the Dispose pattern. The documentation on this in MSDN isn't very good. It stipulates what you should do, but not why you should do it and it's probably why so few developers implement it properly, and that includes me. The authors give an excellent discussion on why should use IDisposable and how you should implement it so there’s no excuse for me doing it incorrectly from now on. So, yes, I really enjoyed this book. It's well designed and written and it works well on two levels, firstly you get an insight into the .net framework, but also it’s a great general good .net practices book. Also it’s well worth checking out Brad Abrams blog.

A simple WSDL object

Wsdl is great, it's like reflection for web services. As I discussed in this post, you can use it to dynamically discover and call web services at runtime. The dot net framework provides a class, 'System.Web.Services.Description.ServiceDescription' that takes a wsdl stream and provides an object model to walk the wsdl. It's really powerfull, but kinda awkward to use if you just want to get a list of web methods with their parameter names and types. Faced with this requirement last week, I put together a simple facade for the ServiceDescription class. You just create a new WebServiceInfo instance using its factory method 'OpenWsdl' and then you can iterate through all the web methods and their parameters and return parameters. Here's the NUnit test which pretty much shows how it works:

[Test]
public void ServiceDescriptionSpike()
{
   string url = "http://localhost:1297/Test/TestService.asmx";

   WebServiceInfo webServiceInfo = WebServiceInfo.OpenWsdl(new Uri(url));

   Console.WriteLine(string.Format("WebService: {0}", webServiceInfo.Url));

   foreach (WebMethodInfo method in webServiceInfo.WebMethods)
   {
       Console.WriteLine(string.Format("\tWebMethod: {0}", method.Name));
       Console.WriteLine("\t\tInput Parameters");

       foreach (Parameter parameter in method.InputParameters)
       {
           Console.WriteLine(
               string.Format("\t\t\t{0} {1}", parameter.Name, parameter.Type));   
       }

       Console.WriteLine("\t\tOutput Parameters");

       foreach (Parameter parameter in method.OutputParameters)
       {
           Console.WriteLine(
               string.Format("\t\t\t{0} {1}", parameter.Name, parameter.Type));
       }
   }
}

And here's the code. As you can see it just wraps the ServiceDescription class, you can see what a pain it is to get this information out of it.

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Text;
using System.Web.Services.Description;
using System.Net;
/// <summary>
/// Information about a web service
/// </summary>
public class WebServiceInfo
{
   WebMethodInfoCollection _webMethods = new WebMethodInfoCollection();
   Uri _url;
   static Dictionary<string, webserviceinfo=""> _webServiceInfos =
       new Dictionary<string, webserviceinfo="">();

   /// <summary>
   /// Constructor creates the web service info from the given url.
   /// </summary>
   /// <param name="url">
   private WebServiceInfo(Uri url)
   {
       if (url == null)
           throw new ArgumentNullException("url");
       _url = url;
       _webMethods = GetWebServiceDescription(url);
   }

   /// <summary>
   /// Factory method for WebServiceInfo. Maintains a hashtable WebServiceInfo objects
   /// keyed by url in order to cache previously accessed wsdl files.
   /// </summary>
   /// <param name="url">
   /// <returns></returns>
   public static WebServiceInfo OpenWsdl(Uri url)
   {
       WebServiceInfo webServiceInfo;
       if (!_webServiceInfos.TryGetValue(url.ToString(), out webServiceInfo))
       {
           webServiceInfo = new WebServiceInfo(url);
           _webServiceInfos.Add(url.ToString(), webServiceInfo);
       }
       return webServiceInfo;
   }

   /// <summary>
   /// Convenience overload that takes a string url
   /// </summary>
   /// <param name="url">
   /// <returns></returns>
   public static WebServiceInfo OpenWsdl(string url)
   {
       Uri uri = new Uri(url);
       return OpenWsdl(uri);
   }

   /// <summary>
   /// Load the WSDL file from the given url.
   /// Use the ServiceDescription class to walk the wsdl and create the WebServiceInfo
   /// instance.
   /// </summary>
   /// <param name="url">
   private WebMethodInfoCollection GetWebServiceDescription(Uri url)
   {
       UriBuilder uriBuilder = new UriBuilder(url);
       uriBuilder.Query = "WSDL";

       WebMethodInfoCollection webMethodInfos = new WebMethodInfoCollection();

       HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(uriBuilder.Uri);
       webRequest.ContentType = "text/xml;charset=\"utf-8\"";
       webRequest.Method = "GET";
       webRequest.Accept = "text/xml";

       ServiceDescription serviceDescription;

       using (System.Net.WebResponse response = webRequest.GetResponse())
       using (System.IO.Stream stream = response.GetResponseStream())
       {
           serviceDescription = ServiceDescription.Read(stream);
       }

       foreach (PortType portType in serviceDescription.PortTypes)
       {
           foreach (Operation operation in portType.Operations)
           {
               string operationName = operation.Name;
               string inputMessageName = operation.Messages.Input.Message.Name;
               string outputMessageName = operation.Messages.Output.Message.Name;

               // get the message part
               string inputMessagePartName =
                   serviceDescription.Messages[inputMessageName].Parts[0].Element.Name;
               string outputMessagePartName =
                   serviceDescription.Messages[outputMessageName].Parts[0].Element.Name;

               // get the parameter name and type
               Parameter[] inputParameters = GetParameters(serviceDescription, inputMessagePartName);
               Parameter[] outputParameters = GetParameters(serviceDescription, outputMessagePartName);

               WebMethodInfo webMethodInfo = new WebMethodInfo(
                   operation.Name, inputParameters, outputParameters);
               webMethodInfos.Add(webMethodInfo);
           }
       }

       return webMethodInfos;
   }

   /// <summary>
   /// Walk the schema definition to find the parameters of the given message.
   /// </summary>
   /// <param name="serviceDescription">
   /// <param name="messagePartName">
   /// <returns></returns>
   private static Parameter[] GetParameters(ServiceDescription serviceDescription, string messagePartName)
   {
       List<parameter> parameters = new List<parameter>();

       Types types = serviceDescription.Types;
       System.Xml.Schema.XmlSchema xmlSchema = types.Schemas[0];

       foreach (object item in xmlSchema.Items)
       {
           System.Xml.Schema.XmlSchemaElement schemaElement = item as System.Xml.Schema.XmlSchemaElement;
           if (schemaElement != null)
           {
               if (schemaElement.Name == messagePartName)
               {
                   System.Xml.Schema.XmlSchemaType schemaType = schemaElement.SchemaType;
                   System.Xml.Schema.XmlSchemaComplexType complexType = schemaType as System.Xml.Schema.XmlSchemaComplexType;
                   if (complexType != null)
                   {
                       System.Xml.Schema.XmlSchemaParticle particle = complexType.Particle;
                       System.Xml.Schema.XmlSchemaSequence sequence = particle as System.Xml.Schema.XmlSchemaSequence;
                       if (sequence != null)
                       {
                           foreach (System.Xml.Schema.XmlSchemaElement childElement in sequence.Items)
                           {
                               string parameterName = childElement.Name;
                               string parameterType = childElement.SchemaTypeName.Name;
                               parameters.Add(new Parameter(parameterName, parameterType));
                           }
                       }
                   }
               }
           }
       }
       return parameters.ToArray();
   }

   /// <summary>
   /// WebMethodInfo
   /// </summary>
   public WebMethodInfoCollection WebMethods
   {
       get { return _webMethods; }
   }

   /// <summary>
   /// Url
   /// </summary>
   public Uri Url
   {
       get { return _url; }
       set { _url = value; }
   }
}

/// <summary>
/// Information about a web service operation
/// </summary>
public class WebMethodInfo
{
   string _name;
   Parameter[] _inputParameters;
   Parameter[] _outputParameters;

   /// <summary>
   /// OperationInfo
   /// </summary>
   public WebMethodInfo(string name, Parameter[] inputParameters, Parameter[] outputParameters)
   {
       _name = name;
       _inputParameters = inputParameters;
       _outputParameters = outputParameters;
   }

   /// <summary>
   /// Name
   /// </summary>
   public string Name
   {
       get { return _name; }
   }

   /// <summary>
   /// InputParameters
   /// </summary>
   public Parameter[] InputParameters
   {
       get { return _inputParameters; }
   }

   /// <summary>
   /// OutputParameters
   /// </summary>
   public Parameter[] OutputParameters
   {
       get { return _outputParameters; }
   }
}

/// <summary>
/// A collection of WebMethodInfo objects
/// </summary>
public class WebMethodInfoCollection : KeyedCollection<string, webmethodinfo="">
{
   /// <summary>
   /// Constructor
   /// </summary>
   public WebMethodInfoCollection() : base() { }

   protected override string GetKeyForItem(WebMethodInfo webMethodInfo)
   {
       return webMethodInfo.Name;
   }
}

/// <summary>
/// represents a parameter (input or output) of a web method.
/// </summary>
public struct Parameter
{
   /// <summary>
   /// constructor
   /// </summary>
   /// <param name="name">
   /// <param name="type">
   public Parameter(string name, string type)
   {
       this.Name = name;
       this.Type = type;
   }

   /// <summary>
   /// Name
   /// </summary>
   public string Name;
   /// <summary>
   /// Type
   /// </summary>
   public string Type;
}

Dynamically creating a web service proxy class

I'm working on a service integration project these days and so I've been having an interesting time getting to grips with web services and all that stuff. (as you can probably see from some of my recent posts). This code project article by Roman Kiss is really cool, its about creating your own loosely coupled web service bus, but what I really liked about it was the bit about creating web service proxy classes on the fly. Basically wsdl.exe, the command line tool that you can use for generating web service proxy classes, wraps a framework class 'ServiceDescriptionImporter'. It takes a 'ServiceDescription' class with a wsdl file loaded in it and returns a code dom namespace. You can compile any code dom namespace using the CSharpCodeProvider, and you can do it in memory and then directly access the methods and types on the generated assembly. That is so cool, it kind of brings the promise of wsdl as a web service reflection language directly into your project.

I love Regex

I'm no expert, and a relatively recent convert to them, but I love regular expressions. They make so many awkward text manipulation tasks easy. Today I had to write a little function to turn a legacy application date into a dotnet DateTime struct. The legacy application's date is in the format CYYMMDD where C is the century, 0 for 1900, 1 for 2000 etc, YY is the two digit year, MM is the month and DD is the day of the month. It's really easy to parse this into a DateTime struct using the dotnet Regex class:

public static DateTime ToDateTime(string geniusDate)
{
    Regex regex = new Regex(@"^(\d)(\d{2})(\d{2})(\d{2})$");
    Match match = regex.Match(geniusDate);
    if (!match.Success)
    {
        throw new ArgumentException("not a valid geniusDate");
    }

    int century = (int.Parse(match.Groups[1].Value) * 100) + 1900;
    int year = int.Parse(match.Groups[2].Value) + century;
    int month = int.Parse(match.Groups[3].Value);
    int day = int.Parse(match.Groups[4].Value);

    return new DateTime(year, month, day);
}

Also I really like the regular expression search and replace built into visual studio. Here's one that changes a field declaration like:

 string _name;

into a property like:

 string name
 {
  get{ return _name; }
  set{ _name = value;}
 }

Just put this:

:b*<{.*}>:b<_{.*}>;

in the 'find what' field, and this:

\1 \2\n{\nget{ return _\2; }\nset{ _\2 = value;}\n}

in the 'replace with' field. Don't forget to check the 'Use' check box and select 'Regular expressions'. It's a shame that the regular expression syntax is different for the visual studio and the Regex class. The captures are () in Regex, but {} in visual studio. Now, of course it's much easier to type control R, E and let the refactoring tools do this job for you, but it's a good demonstration of how much you can do with search and replace regular expressions.

The world's most misunderstood programming language

I really enjoyed reading this post by Douglas Crockford, 'The world's most misunderstood programming language'. I have always dismissed JavaScript as a kind of toy language that I've more suffered than enjoyed working with. I never realised it was so powerfull, probably for all the reasons that Mr Crockford enumerates in his article. I've been very interested in object oriented programming for a long time now, but I'd never heard of stuff like 'closures', 'lamda expressions' or 'functional programming' until recently. JavaScript it turns out is like 'Lisp in C syntax' with all those cool features. I guess this is all bourne out by the current interest in AJAX and more powerfull browser based apps.

Making raw web service calls with the HttpWebRequest class

Sometimes it's really nice to be able to make a raw call to a web service by manaully putting together your own SOAP envelope. The System.Net.HttpWebRequest class makes it really easy. There are a number of good reasons to do this. Maybe you want to call a web service without having to create a proxy class with wsdl.exe, or maybe you want to have more control over the creation of the SOAP envelope or you don't want to have to rely on the XmlSerializer. Maybe you want to create the xml by doing xslt transforms rather than serializing a .net type. First you need to create the envelope xml, this function takes a string of xml data as the content and inserts it into a soap envelope. Just open your web service in IE by typing the url into the Address bar to see what the structure of the soap:Body should be.

    static string _soapEnvelope =
@"<soap:Envelope
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xmlns:xsd='http://www.w3.org/2001/XMLSchema'
    xmlns:soap='http://schemas.xmlsoap.org/soap/envelope/'>
<soap:Body></soap:Body></soap:Envelope>";

    private static XmlDocument CreateSoapEnvelope(string content)
    {
        StringBuilder sb = new StringBuilder(_soapEnvelope);
        sb.Insert(sb.ToString().IndexOf("</soap:Body>"), content);

        // create an empty soap envelope
        XmlDocument soapEnvelopeXml = new XmlDocument();
        soapEnvelopeXml.LoadXml(sb.ToString());

        return soapEnvelopeXml;
    }

Next you create the HttpWebRequest object. The url is the url of the .aspx file and the action is your namespace plus the web method, e.g: 'mikehadlow.com/adder'.

    private static HttpWebRequest CreateWebRequest(string url, string action)
    {
        HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
        webRequest.Headers.Add("SOAPAction", action);
        webRequest.ContentType = "text/xml;charset=\"utf-8\"";
        webRequest.Accept = "text/xml";
        webRequest.Method = "POST";
        return webRequest;
    }

Insert the envelope into the web request.

    private static void InsertSoapEnvelopeIntoWebRequest(XmlDocument soapEnvelopeXml, HttpWebRequest webRequest)
    {
        using (Stream stream = webRequest.GetRequestStream())
        {
            soapEnvelopeXml.Save(stream);
        }
    }

Then you can call .GetResponse on the HttpWebRequest object to call the web service. You get the response back by getting the response stream with the GetResponseStream method of the webResponse object. Here's it all put together:

    static string _url = "http://mikehadlow.com/myService.asmx";
    static string _action = "http://mikehadlow.com/myWebMethod";
    static string _inputPath = @"C:\mike\Play\input.xml";
    static string _outputPath = @"C:\mike\Play\output.xml";

    static void Main(string[] args)
    {
        string content = File.ReadAllText(_inputPath);
        XmlDocument soapEnvelopeXml = CreateSoapEnvelope(content);
        HttpWebRequest webRequest = CreateWebRequest(_url, _action);
        InsertSoapEnvelopeIntoWebRequest(soapEnvelopeXml, webRequest);

        // begin async call to web request.
        IAsyncResult asyncResult = webRequest.BeginGetResponse(null, null);

        // suspend this thread until call is complete. You might want to
        // do something usefull here like update your UI.
        asyncResult.AsyncWaitHandle.WaitOne();

        // get the response from the completed web request.
        string soapResult;
        using (WebResponse webResponse = webRequest.EndGetResponse(asyncResult))
        using (StreamReader rd = new StreamReader(webResponse.GetResponseStream()))
        {
            soapResult = rd.ReadToEnd();
        }
        File.WriteAllText(_outputPath, FormatDocument(soapResult));
    }

Intellisense isn't always right

I've found myself becoming almost too reliant on intellisense in VS2005. I almost didn't do the obvious thing today, just because intellisense didn't understand it. I wanted to increment the last two digits of a reference string. It's looks something like: "UKDRNY500011106" and I needed to change it to: "UKDRNY500011107" so I wanted to write this

return policyRef.Substring(0, 12) + (int.Parse(policyRef.Substring(12, 2)) + 1).ToString("00");

If you try and type that line of code into VS, when you get to the '.' before ToString, intellisense doesn't suggest 'ToString' or anything. It doesn't recognise that the expression in brackets is an integer value. But if you type it anyway, it compiles and runs just fine. The silly thing is that I almost didn't write it because intellisense didn't suggest it.

Where do role-based security checks go in my Application?

I recently had to help a team who had problems with their role based security implementation. They weren’t sure where to locate role based security checks in their application. Their current design wasn't working. They held users, roles, use cases and method names in a database. The method names described web service methods and they mapped onto particular use cases that applied to particular roles. Each user was assigned a role. This way they could do a lookup as each method was called and find out if the logged on user was allowed to run it. If they weren't, the web service threw an exception, which the application caught and used to alert the user that they were doing something illegal. Of course, it's better to enforce role-based rules at the UI. A button that the user isn't allowed to click should be invisible to them, or disabled, rather than allowing them to click it and get an error message. To that end, the other part of their security mechanism linked use-cases to control names, so that controls could be enabled or disabled depending on whether the user belonged to a role assigned to that control's use case. There were several problems with this model. The first and most obvious one was that there were two different mechanisms to do the same thing; the method access based checks and the control-based checks. Both enforced access rules to certain bits of functionality, but had two different implementations and two different schemas in their security database. There was ample opportunity for support people to configure conflicting rules for the UI and the web services. The second problem, and probably a more fundamental one was the assumption that use cases mapped exactly to web service method calls. For example, my method call, 'Save Customer', would map to a particular use case. But what if one use case said that only an admin person could change a customer's credit status, while another one said that a clerical user could change a customer's address. Does it make sense in our application design to split Save Customer into several use case based calls? Probably not. There was a real danger that the security design would have driven out a very awkward physical implementation. A further assumption was that they wouldn't have data related security requirements. For example, say that the accounts department could only update one type of customer, but another type could only be updated by the call center. The customer type is defined by a property of customer, but of course they both use the same Save Customer call. And imagine the customer types can be user defined, there's no way you can implement that with a static security model. I think the fundamental problem with the whole approach was not seeing role based security roles for what they are: business rules. Just like a customer's credit limit, whether a user of a particular role can edit a customer is a business rule. Like all business rules in our Domain Driven application design, the role based security checks should be implemented in our Business Entities (Business Objects, or whatever you call them). If the rules are expressed as properties of our domain entities, Customer.CanEdit, for example, you can both bind them to the UI, by binding the CanEdit property to the Save button's Enabled property, and apply them at lower levels of the application architecture. You could query Customer.CanEdit in your Data Access Layer before running the customer update, or query the same property in your web service. Putting rule based security checks in your domain objects also makes sense from the point of view of application design. Since they are business rules they sit nicely alongside the other rules that apply to a particular entity and can interact with them if need be. You can code rules that maybe intersect a user rule, a property of the entity and another entity. Best of all you can do it all in the same place.