How Brokers Build Publish/Subscribe

The broker we built gives each subscriber its own queue and copies each message to it. That is one of three common ways brokers build publish/subscribe:

• A queue for each subscriber
• A log
• Live delivery only

A queue for each subscriber

This is the way we use for Shopend. The broker copies each message to every queue on the topic’s list, and each subscriber takes messages from its own queue. RabbitMQ works this way: a fanout exchange copies each message to every queue bound to it. So do Amazon SNS and SQS: each SQS queue subscribed to an SNS topic gets its own copy of every message.

A log

In the second way, the broker keeps every message of a topic in order, for a set time. Reading a message does not remove it. Each subscriber keeps its own position in the log and moves it forward as it reads:

graph LR
    subgraph L[order-placed log]
        direction LR
        M91[order 91] --- M92[order 92] --- M93[order 93] --- M94[order 94] --- M95[order 95]
    end
    N[Notification service] -.->|position: order 95| M95
    A[Analytics service] -.->|position: order 92| M92

Here, the notification service has read up to order 95, and the analytics service is behind, at order 92. Neither position affects the other.

Apache Kafka works this way. Kafka is an open-source broker, first built at LinkedIn and released in 2011. By default, it keeps a topic’s messages for a week. Kafka calls its subscribers consumer groups. Kafka keeps the messages in order within each part of a topic, called a partition. How Kafka splits a topic into partitions is beyond the scope of this course.

Because the log keeps old messages, a subscriber added later can start from the oldest message the log still keeps. By default, Kafka starts a new subscriber from the newest message, so the subscriber has to be set up to start from the oldest.

Live delivery only

In the third way, the broker sends each message to the subscribers connected at that moment. It keeps nothing. A subscriber that is not connected when a message is published never gets it.

Redis, which we met as a cache server, works this way with its PUBLISH and SUBSCRIBE commands.

Which way works

Live delivery only is the simplest of the three. Let’s check whether it works for three cases.

Case 1: The shipping service is restarted for a new release while orders are being placed. It must still ship every order. Live delivery only does not work here. Orders published while the shipping service restarts would never reach it. We need a way that keeps messages: a queue for each subscriber, which we use, or a log.

Case 2: The analytics service starts next month, and it wants to count the orders of the week before it started.** Only a log works here. A queue holds only the messages published after the queue was added to the topic’s list. A log that keeps a week of messages lets the analytics service start a week back. A log can keep messages for as long as we set it to, by time or by size. A longer time means more storage.

Case 3: Courselore’s instances pass each new chat message to each other, so each can send it on its own connections. Live delivery only works here. The channel between Courselore’s instances is built with Redis. Each chat message is saved in the database before it is published. An instance that is restarting has no open connections, so it has no client to send the message to. When its users’ clients reconnect, they read the room’s messages from the database.