Instances of a Subscriber

So far, we have followed one instance of each service. In practice, any of Shopend’s services can run as several instances, as the commerce service does. Suppose the notification service runs as three instances. Should each instance get its own copy of the message for order 93?

A copy for each instance

No. If each instance got a copy, the customer would get three confirmation emails for order 93.

To prevent that, the notification service would have to store a record of each email it sent, and each instance would check the record before it sends an email. This has two costs. First, two instances can check at the same time, both find no record, and both send the email. So checking and storing the record must happen in one transaction that claims the email for one instance. Second, every instance handles every message, even though another instance sends the email for most of them.

Competing consumers

Instead, the three instances share the email queue. When an instance is ready, it takes the next message off the queue. The message is then gone from the queue, so no other instance can get it. So each message goes to one instance, and the instances do not need to coordinate.

graph TD
    T[order-placed topic] -->|copy| EQ[Email queue]
    T -->|copy| SQ[Shipping queue]
    EQ -->|order 93| N1[Notification instance 1]
    EQ -->|order 94| N2[Notification instance 2]
    EQ -->|order 95| N3[Notification instance 3]
    SQ --> Sh[Shipping service]

Several consumers that take messages from one queue, with each message going to one of them, are called competing consumers. The name comes from Enterprise Integration Patterns (2003), a book by Gregor Hohpe and Bobby Woolf. The same book calls a queue a point-to-point channel and a topic a publish-subscribe channel.

The instances do not handle the messages in the order they were published. Three instances send three emails at the same time. Each email is sent on its own, so the order does not matter here.

To send emails faster, we start more instances on the email queue. The order worker and the broker do not change, and each new instance takes its share of the messages.

The broker and a load balancer

Recall from scaling the API that a load balancer spreads requests across instances, and each request goes to one instance. The broker also spreads messages across instances, one message to each. How is it different from a load balancer?

The two differ in who decides which instance gets the work. A load balancer picks an instance for each request and sends the request there, even if that instance is busy. The broker does not pick. Each instance takes the next message from the queue when it is ready. A slow instance takes fewer messages, and a burst of messages waits in the queue instead of going to busy instances.

Amazon SQS works this way, because a consumer sends SQS a request for messages. RabbitMQ sends messages to its consumers over their connections, so there the broker does pick. RabbitMQ has a setting called prefetch that limits how many messages a consumer can hold at a time. With prefetch set to one message, RabbitMQ behaves the same way as SQS.

When every instance needs a copy

Sometimes every instance of a service needs every message. Recall Courselore from the real-time chat chapters. Courselore is a monolith: it runs as one application on one server. Its chat sends new messages to users over WebSockets, and the server holds each user’s open connection. One server can hold all of Courselore’s connections.

If one server could no longer hold all of Courselore’s connections, we would run copies of the whole application, two instances behind a load balancer. Each user’s WebSocket is open on one instance, and only that instance can send on it. Suppose Marcus’s connection is open on Instance 2. Priya posts a message, and the load balancer sends her POST to Instance 1. Instance 1 saves the message and sends it on its own connections. Marcus never gets it.

sequenceDiagram
    participant P as Priya's client
    participant A as Instance 1
    participant B as Instance 2
    participant M as Marcus's client
    P->>A: POST /rooms/12/messages
    Note over A: sends it on Instance 1's connections only
    Note over B,M: Marcus never gets it

The instances need a channel between them, so that every new message reaches every instance, and each instance can send it on its own connections. That channel is a topic for new chat messages. Each instance publishes every new message to the topic, and each instance subscribes to the topic on its own. Each instance sends the copies it receives on its own connections. That includes the copy of a message it published itself. So Instance 2 gets a copy of Priya’s message and sends it to Marcus.

The WebSockets chapter named Redis as a common choice for this channel.

Two ways to subscribe

So there are two ways for the instances of a service to subscribe:

• The three notification instances share one queue. They act as one subscriber, and each email is sent once.
• Courselore’s two instances subscribe on their own. Each instance is a subscriber, and each one gets every message.

Which way to use depends on the work. An email must be sent once, by any instance, so the notification instances share a queue. A chat message must be sent on every connection, and only the instance that holds a connection can send on it, so every Courselore instance needs every message.