Wednesday, April 13, 2011

Serializing Continuations

Continuations are cool. They allow you to write some code that uses the enclosing state, but which can be executed in some other context at some other time. What if we could serialize the continuation and put it away in a database, a file, or even a message on a service bus only to be grabbed later and then executed? Well we can.

Take a look at this code:

private const string filePath = @"C:\temp\serialized.action";

public static void BuryIt()
{
const string message = "Hello!";

var bytes = DoSomethingLater(name => Console.WriteLine(message + " " + name));

using(var stream = File.Create(filePath))
{
stream.Write(bytes, 0, bytes.Length);
}
}

public static void ResurrectIt()
{
var bytes = File.ReadAllBytes(filePath);
ExecuteSerializedAction("Mike", bytes);
}

public static byte[] DoSomethingLater(Action<string> action)
{
var formatter = new BinaryFormatter();
var stream = new MemoryStream();

formatter.Serialize(stream, action);
stream.Position = 0;
return stream.GetBuffer();
}

public static void ExecuteSerializedAction(string name, byte[] serializedAction)
{
var formatter = new BinaryFormatter();
var stream = new MemoryStream(serializedAction);
var action = (Action<string>)formatter.Deserialize(stream);

action(name);
}

In ‘BuryIt’ we call DoSomethingLater passing to it a continuation that includes some of TryIt’s state, namely the message string “Hello!”. DoSomethingLater takes takes the ‘action’ delegate, serializes it and returns the serialized bytes to its caller. BuryIt then writes the bytes to a file.

Next we execute ResurrectIt. It opens the file, reads the bytes and  hands them to ExecuteSerializedAction. This de-serializes the byte array back to an Action<string> and then executes it with the name parameter. When you execute it, it prints out:

Hello! Mike

This is very cool. Say we had some long running process where we wanted to execute each continuation at an interval  and have the process go away in between each execution. We could use this technique to store the state at each stage in a database and have each instance carry on where the previous one left off.

19 comments:

  1. Hi Mike

    That's pretty cool, I didn't realise they could be serialised.

    Does it work with any "delegate". For instance will it still work if the "Hello!" string were not a constant and it was garbage collected?

    ReplyDelete
  2. Oh yes. The whole process can go away. You can bring it back up later and run the continuation.. try it :)

    ReplyDelete
  3. Very interesting. The threat model implications are making my head hurt.

    ReplyDelete
  4. Good stuff Mike.

    My background is with interpreters and automation. And for for the past few years now I've been toying with the idea that all code is in actual fact data, and that execution is really just a view on said data. It's not clear in my head yet, but there's not one class of problem I've come across where this model doesn't seem to work. Thanks for helping my codify my thoughts a bit more. ho hum...

    ReplyDelete
  5. Chris Martin8:10 am

    Very cool, Mike. I never realized you could do this either.

    ...but...memorystream.ToArray() brutha. :)

    ReplyDelete
  6. That's pretty awesome. Never thought about using delegates and serialization together!

    ReplyDelete
  7. This is pretty cool, I admit. However, given issues around versioning and custom types, would something more like Workflow Foundation not be more appropriate?

    I'm not a fan of WF myself, but just wanted to ask the question. :)

    ReplyDelete
  8. Interesting, but is that a continuation? After all you are giving your delegate a new state after resurrection. Shouldn't continuation capture stack state?


    Btw, have you looked into Mono.Tasklets?

    ReplyDelete
  9. Srdjan, yup, I'm doing both. the continuation captures the message value, but then gets name when it's run.

    ReplyDelete
  10. oh, yes "Hello" was captured... Cool!

    ReplyDelete
  11. lift (the scala web framework) works this way... gr8 perf but it does mean JVM affinity...

    ReplyDelete
  12. Anonymous9:21 pm

    Very interesting. However, serialization fails if 'message' is passed as a parameter rather than being a constant, which would be tons more useful.

    This is what I mean. Change BuryIt to

    public static void BuryIt(string message)
    {
    ...
    }

    ReplyDelete
  13. Anonymous, you're right, the serializer complains that the closed over variable isn't marked as serializable.

    That's a pity.

    ReplyDelete
  14. Cool technique!

    But I think you'd need to be very careful about versioning your assembly.

    Remember that anonymous types get created when you use closures. If in a future version of the assembly you capture different variables in the closure that type is going to change.

    So if you try to load a serialized delegate from a previous version of the assembly into a process running the new version, the BinaryFormatter won't be able to find that old type, and it will fail.

    ReplyDelete
  15. Should have read @Anonymous comment first. So you can't use full blown closures anyway.

    But another case that might be a problem is where you change the parameters to the delegate in the new version of the assembly. That means that the old anonymous method will no longer exist, and that will cause a failure when de-serializing.

    ReplyDelete
  16. This could be stored away in RavenDb and pulled out later with no fuss! I like it!

    ReplyDelete
  17. Dave-O6:11 pm

    I love the simplicity of this API - I'd love to see it further developed for things like allowing use of routing keys etc

    ReplyDelete
  18. Any idea if we'll ever see CLR-level continuation serialization?

    ReplyDelete
  19. So instead of calling a remote client with some data and retrieve the result, you could call a remote client and get the way of calculation(descriptor), cache it and do the calculation whenever necessary by yourself. (if cache is up to date)

    Different kind of client server communication. :)
    (limited to .net, and maybe to fw version. building blocks of a description of a function may vary between fw versions, i do not know.)

    Will try it :)

    ReplyDelete

Note: only a member of this blog may post a comment.