Friday, July 24, 2009

How MassTransit Publish and Subscribe works

clip_image002

This is a follow-on from my last post, A First Look at MassTransit. Here’s my take on how publish and subscribe works. It’s based on a very brief scan of the MT code, so there could well be misunderstandings and missing details.

The core component of MassTransit is the ServiceBus, it’s the primary API that services use to subscribe to and publish messages. The ServiceBus has an inbound and outbound pipeline. When publish is called, a message gets sent down the pipeline until a component that cares that message type dispatches it to an endpoint. Similarly, when a message is received it is passed down the input pipeline giving each component a chance to process it.

Understanding how the input and output pipelines are populated is the key to understanding how MassTransit works. It’s instructive to get a printout of your pipelines by inspecting them with the PipelineViewer. I’ve created a little class to help with this:

 

using System.IO;
using MassTransit.Pipeline;
using MassTransit.Pipeline.Inspectors;

namespace MassTransit.Play.Helpers
{
    public class PipelineWriter : IPipelineWriter
    {
        private readonly TextWriter writer;
        private readonly IServiceBus bus;

        public PipelineWriter(IServiceBus bus, TextWriter writer)
        {
            this.bus = bus;
            this.writer = writer;
        }

        public void Write()
        {
            writer.WriteLine("InboundPipeline:\r\n");
            WritePipeline(bus.InboundPipeline);
            writer.WriteLine("OutboundPipeline:\r\n");
            WritePipeline(bus.OutboundPipeline);
        }

        private void WritePipeline(IPipelineSink<object> pipeline)
        {
            var inspector = new PipelineViewer();
            pipeline.Inspect(inspector);
            writer.WriteLine(inspector.Text);
        }
    }
}

Let’s look at the sequence of events when RuntimeServices.exe, a subscriber service and a publishing service start up.

When RuntimeServices starts up the SubscriptionService creates a list of ‘SubscriptionClients’. Initially this is empty.

clip_image003

When our subscriber comes on line, it sends an AddSubscriptionClient message to the subscription service. The subscription service then adds our subscriber to its list of subscription clients.

clip_image004

Next our publisher comes on line. It also sends an AddSubscriptionClient message to the subscription service. It too gets added to the subscription clients list.

clip_image005

When the subscriber subscribes to a particular message type, ServiceBus sends an AddSubscription message to SubscriptionService which in turn scans its list of subscription clients and sends the AddSubscription message to each one.

SubscriptionService also adds the subscription to its list of subscriptions so that when any other services come on line it can update them with the list.

The publisher receives the AddSubscription message that was broadcast to all the subscription clients and adds the subscriber endpoint to its outbound pipeline. Note that the Subscriber also receives it’s own AddSubscription message back and adds itself to its outbound pipeline (not shown in the diagram).

clip_image006

The subscriber also adds a component to its inbound pipeline to listen for messages of the subscribed type. I haven’t show this in the diagram either.

When the publisher publishes a message, it sends the message down its outbound pipeline until it is intercepted by the subscriber’s endpoint and dispatched to the subscriber’s queue. The subscription service is not involved at this point.

clip_image007

I hope this is useful if you’re trying to get to grips with MassTransit. Thanks to Dru for clarifying some points for me.

6 comments:

Unknown said...

nice!!

Chris said...

It's worth mentioning that all of the services (subscription, health, and timeout) all use the state machine based saga support from MassTransit/Magnum to manage subscription and subscriptionclient state. Trying to implement all of those services as sagas really helped flesh out the saga DSL.

Excellent write-up BTW.

Mike Hadlow said...

Dru, Chris, thanks guys. I plan to blog about sagas next. Once again, thanks for providing an excellent framework.

Anonymous said...

Thanks Mike,

I have a doubt about service bus in general and MassTransit in particular.Could you help me?

Isnt possible work in disconnect mode? I refer about send mesages without consumers.
For example, I want to send emails and I use Masstransit for it.Then I send a message with email,body etc inside but I want work with Amazon Ec2.Then, every 6 hours I pick up a ec2 instance and send all pending mails.

But this is not possible with service bus because the publisher need know the consumers before send a message, is it true?
For me its a very connected mode, not support crash very well.

I did test with yor code and when the consumer is down, the server (the queues) store the messages (durable message,http://www.enterpriseintegrationpatterns.com/DurableSubscription.html) but is necessary in the principle that the consumer is connected.

Thanks.

Mike Hadlow said...

Hi Anonymous,

You are correct in saying that the subcriber needs to have registered with the bus in order for it to recieve published messages. However, once a subscriber is registered, it can be unreliable. MSMQ is a robust store-and-forward messaging system, so if the subscribing server goes off-line, it won't loose the messages but will simply recieve them once it comes back on-line.

In your scenario you would have to bring up your EC2 instance, subscribe to your email message, then close down the EC2 instance. You can later bring the EC2 instance back up to fire off all the queued messages.

Anonymous said...

ok, thanks a lot Mike