Wes Dyer talking about LINQ

More browsing today on MSDN lead me to an interview with Wes Dyer taking about LINQ. He's one of the guys on the C# compiler team. A very bright chap indeed and he gives one of the best explanations of how LINQ works that I've yet seen. The white board session especially in the second half of the video is a fantastic mind expanding ride and shows how LINQ is far far more than just queries. Typically I think Microsoft's marketing have probably muddied the waters again by calling it 'language integrated query' because, although querying is something you can do with it, it's much more like 'functional programming in C#'. For example, Wes showed how you can replace typical looping patterns with LINQ 'queries'. Take this little code snippet to get the even numbers from an array of numbers:

int[] list = int[] {1,2,3,4,5,6,7};
List<int> evenNumbers = new List<int>();
foreach(int n in list)
{
if(n % 2 == 0)
 evenNumbers.Add(n);
}

You can write the same thing with LINQ:

int[] list = int[] {1,2,3,4,5,6,7};
var evenNumbers =
from n in list
where n % 2 == 0
select n;

I think it's much easier to read the intention of the LINQ version. One thing that really struck me was how they use the custom iterators that I talked about in my last post to create the 'where' and 'select' clauses in LINQ. A very bright light bulb went off in my head at that point! Wes Dyer's blog is also well worth checking out.

Creating set based filters with custom iterators

Well it's back to work after a very nice two week break. I wrapped up my last contract with Ace Insurance in Crawley (hi guys!) three weeks ago, and I've been at my new assignment for a week now. I've been very busy getting up to speed with my new role (doing yet more application integration), but I did have time to skim MSDN where I came across this great article by Bill Wagner, titled 'Custom Iterators'. He talks about custom iterators, predicates and generics, all stuff that I'm quite familiar with, but it's the way he puts them together to create aggregate set functions, just like the ones that I've been discovering with functional languages like F#, that makes it really interesting. Custom iterators are new in C# 2.0 and make it really easy to create an instance of IEnumerable. Instead of implementing the entire interface, you just have to write a method that returns IEnumerable and has a yield statement for each item you want your IEnumerable to return. Behind the scenes the C# compiler creates an instance of IEnumerable for you by turning your method into a class with fields for all your local variables. For example, if you define an iterator that returns the numbers one to ten:

IEnumerable<int> OneToTen()
{
   for(int i=1; i<=10; i++)
   {
       yield return i;
   }
}

It will return an object of a class that you can imagine has a private field i and implements the IEnumerable.GetEnumerator() interface. You can then use the IEnumerator that it returns just like any other IEnumerator, for example:

public Test()
{
   IEnumerator<int> oneToTen = OneToTen().GetEnumerator();
   oneToTen.MoveNext();
   Console.WriteLine("{0}", oneToTen.Current);
   oneToTen.MoveNext();
   Console.WriteLine("{0}", oneToTen.Current);
}

Will output:

1
2

Now, the trick that Bill Wagner demonstrates, that hadn't occurred to me before is that you can also pass an instance of IEnumerable to your customer iterator to make a filters, transformers or any other set based operation you can think of. Here's a simple filter that only passes even numbers:

IEnumerable<int> Even(IEnumerable<int> list)
{
   foreach(int number in list)
   {
       if(number % 2 == 0)
           yield return number;
   }
}

public Test()
{
   foreach(int number in Even(OneToTen())
   {
       Console.WriteLine("{0}", number);
   }
}

Will output:

2
4
6
8
10

Bill's next trick to use anonymous delegates and generics to write a generic filter:

delegate bool Test<T>(T value);
IEnumerable<T> Filter<T>(IEnumerable<T> list, Test<T> test)
{
   foreach(T item in list)
   {
       if(test(item))
           yield return item;
   }
}

public Test()
{
   IEnumerable<int> evenNumbers = Filter<int>(OneToTen(), delegate(int number)
   {
       return number % 2 == 0;
   });
   foreach(int number in evenNumbers)
   {
       Console.WriteLine("{0}", number);
   }
}

Will output:

2
4
6
8
10

Very neat, but a bit long winded syntax wise I think. This will all change with C# 3.0's lambda expressions. It's yet another step towards a more functional programming style that's such a powerful addition to the programmer's toolbox.