Wednesday, April 13, 2011

Serializing Continuations

Continuations are cool. They allow you to write some code that uses the enclosing state, but which can be executed in some other context at some other time. What if we could serialize the continuation and put it away in a database, a file, or even a message on a service bus only to be grabbed later and then executed? Well we can.

Take a look at this code:

private const string filePath = @"C:\temp\serialized.action";

public static void BuryIt()
{
const string message = "Hello!";

var bytes = DoSomethingLater(name => Console.WriteLine(message + " " + name));

using(var stream = File.Create(filePath))
{
stream.Write(bytes, 0, bytes.Length);
}
}

public static void ResurrectIt()
{
var bytes = File.ReadAllBytes(filePath);
ExecuteSerializedAction("Mike", bytes);
}

public static byte[] DoSomethingLater(Action<string> action)
{
var formatter = new BinaryFormatter();
var stream = new MemoryStream();

formatter.Serialize(stream, action);
stream.Position = 0;
return stream.GetBuffer();
}

public static void ExecuteSerializedAction(string name, byte[] serializedAction)
{
var formatter = new BinaryFormatter();
var stream = new MemoryStream(serializedAction);
var action = (Action<string>)formatter.Deserialize(stream);

action(name);
}

In ‘BuryIt’ we call DoSomethingLater passing to it a continuation that includes some of TryIt’s state, namely the message string “Hello!”. DoSomethingLater takes takes the ‘action’ delegate, serializes it and returns the serialized bytes to its caller. BuryIt then writes the bytes to a file.

Next we execute ResurrectIt. It opens the file, reads the bytes and  hands them to ExecuteSerializedAction. This de-serializes the byte array back to an Action<string> and then executes it with the name parameter. When you execute it, it prints out:

Hello! Mike

This is very cool. Say we had some long running process where we wanted to execute each continuation at an interval  and have the process go away in between each execution. We could use this technique to store the state at each stage in a database and have each instance carry on where the previous one left off.

19 comments:

Unknown said...

Hi Mike

That's pretty cool, I didn't realise they could be serialised.

Does it work with any "delegate". For instance will it still work if the "Hello!" string were not a constant and it was garbage collected?

Mike Hadlow said...

Oh yes. The whole process can go away. You can bring it back up later and run the continuation.. try it :)

Unknown said...

Very interesting. The threat model implications are making my head hurt.

preet sangha said...

Good stuff Mike.

My background is with interpreters and automation. And for for the past few years now I've been toying with the idea that all code is in actual fact data, and that execution is really just a view on said data. It's not clear in my head yet, but there's not one class of problem I've come across where this model doesn't seem to work. Thanks for helping my codify my thoughts a bit more. ho hum...

Chris Martin said...

Very cool, Mike. I never realized you could do this either.

...but...memorystream.ToArray() brutha. :)

Jef Claes said...

That's pretty awesome. Never thought about using delegates and serialization together!

Neil Barnwell said...

This is pretty cool, I admit. However, given issues around versioning and custom types, would something more like Workflow Foundation not be more appropriate?

I'm not a fan of WF myself, but just wanted to ask the question. :)

Srdjan said...

Interesting, but is that a continuation? After all you are giving your delegate a new state after resurrection. Shouldn't continuation capture stack state?


Btw, have you looked into Mono.Tasklets?

Mike Hadlow said...

Srdjan, yup, I'm doing both. the continuation captures the message value, but then gets name when it's run.

Srdjan said...

oh, yes "Hello" was captured... Cool!

Mick Delaney said...

lift (the scala web framework) works this way... gr8 perf but it does mean JVM affinity...

Anonymous said...

Very interesting. However, serialization fails if 'message' is passed as a parameter rather than being a constant, which would be tons more useful.

This is what I mean. Change BuryIt to

public static void BuryIt(string message)
{
...
}

Mike Hadlow said...

Anonymous, you're right, the serializer complains that the closed over variable isn't marked as serializable.

That's a pity.

Unknown said...

Cool technique!

But I think you'd need to be very careful about versioning your assembly.

Remember that anonymous types get created when you use closures. If in a future version of the assembly you capture different variables in the closure that type is going to change.

So if you try to load a serialized delegate from a previous version of the assembly into a process running the new version, the BinaryFormatter won't be able to find that old type, and it will fail.

Unknown said...

Should have read @Anonymous comment first. So you can't use full blown closures anyway.

But another case that might be a problem is where you change the parameters to the delegate in the new version of the assembly. That means that the old anonymous method will no longer exist, and that will cause a failure when de-serializing.

wayne-o said...

This could be stored away in RavenDb and pulled out later with no fuss! I like it!

Dave-O said...

I love the simplicity of this API - I'd love to see it further developed for things like allowing use of routing keys etc

Nathanael Jones said...

Any idea if we'll ever see CLR-level continuation serialization?

Unknown said...

So instead of calling a remote client with some data and retrieve the result, you could call a remote client and get the way of calculation(descriptor), cache it and do the calculation whenever necessary by yourself. (if cache is up to date)

Different kind of client server communication. :)
(limited to .net, and maybe to fw version. building blocks of a description of a function may vary between fw versions, i do not know.)

Will try it :)