Serializing Lua Coroutines With Pluto

In my last post I showed how it’s possible to use Lua as a distributed workflow scripting language because of its build in support for coroutines. But in order to create viable long-running workflows we have to have some way of saving the script’s state at the point that it pauses; we need a way to serialize the coroutine.

Enter Pluto:

“Pluto is a heavy duty persistence library for Lua. Pluto is a library which allows users to write arbitrarily large portions of the "Lua universe" into a flat file, and later read them back into the same or a different Lua universe.”

I downloaded the Win32 build of Tamed Pluto from here and placed it in a directory alongside my Pluto.dll. You have to tell Lua where to find C libraries by setting the Lua package.cpath:

using (var lua = new Lua())
{
    lua["package.cpath"] = @"D:\Source\Mike.DistributedLua\Lua\?.dll";

    ...
}

Lua can now find the Pluto library and we can import it into our script with:

require('pluto')

Here’s a script which defines a function foo which calls a function bar. Foo is used to create a coroutine. Inside bar the coroutine yields. The script uses Pluto to serialize the yielded coroutine and saves it to a file.

-- import pluto, print out the version number 
-- and set non-human binary serialization scheme.
require('pluto')
print('pluto version '..pluto.version())
pluto.human(false)

-- perms are items to be substituted at serialization
perms = { [coroutine.yield] = 1 }

-- the functions that we want to execute as a coroutine
function foo()
    local someMessage = 'And hello from a long dead variable!'
    local i = 4
    bar(someMessage)
    print(i)
end

function bar(msg)
    print('entered bar')
    -- bar runs to here then yields
    coroutine.yield()
    print(msg)
end

-- create and start the coroutine
co = coroutine.create(foo)
coroutine.resume(co)

-- the coroutine has now stopped at yield. so we can
-- persist its state with pluto
buf = pluto.persist(perms, co)

-- save the serialized state to a file
outfile = io.open(persistCRPath, 'wb')
outfile:write(buf)
outfile:close()

This next script loads the serialized coroutine from disk, deserializes it, and runs it from the point that it yielded:

-- import pluto, print out the version number 
-- and set non-human binary serialization scheme.
require('pluto')
print('pluto version '..pluto.version())
pluto.human(false)

-- perms are items to be substituted at serialization
-- (reverse the key/value pair that you used to serialize)
perms = { [1] = coroutine.yield }

-- get the serialized coroutine from disk
infile, err = io.open(persistCRPath, 'rb')
if infile == nil then
    error('While opening: ' .. (err or 'no error'))
end

buf, err = infile:read('*a')
if buf == nil then
    error('While reading: ' .. (err or 'no error'))
end

infile:close()

-- deserialize it
co = pluto.unpersist(perms, buf)

-- and run it
coroutine.resume(co)

When we run the scripts, the first prints out ‘entered bar’ and then yields:

LUA> pluto version Tamed Pluto 1.0
LUA> entered bar

The second script loads the paused ‘foo’ and ‘bar’ and continues from the yield:

LUA> pluto version Tamed Pluto 1.0
LUA> And hello from a long dead variable!
LUA> 4

Having the ability to run a script to the point at which it starts a long running call, serialize its state and store it somewhere, then pick up that serialized state and resume it at another place and time is a very powerful capability. It means we can write simple procedural scripts for our workflows, but have them execute over a distributed service oriented architecture.

The complete code for this experiment is on GitHub here.