Message Order

The shipping queue now gets two kinds of message about order 93. A copy of order-placed tells the shipping service to ship the order. A copy of order-cancelled tells it to stop the shipment. The order is placed before it is cancelled, so order-placed reaches the queue first.

One shipping instance

So far, we have not decided what the shipping service stores. I store one document for each shipment, with a status. When the shipping service handles order-placed, it saves a shipment for order 93 with the status READY:

{ "shopId": "mugshop", "orderId": "93", "status": "READY" }

A shipment that is READY is packed and sent. To stop it, the shipping service sets its status to STOPPED when it handles order-cancelled.

Suppose the broker gives out the messages in a queue in the order they arrived, and the shipping service runs as one instance. The instance handles order-placed first and saves the shipment. Then it handles order-cancelled and stops the shipment. The parcel for order 93 is not sent.

Several shipping instances

Recall that the instances of the notification service share the email queue, and they do not handle the messages in the order they were published. For emails, the order does not matter. Now suppose the shipping service runs as several instances that share the shipping queue in the same way, and the shopper cancels order 93 a few seconds after it is placed:

  1. Instance A takes order-placed for order 93 and crashes before it saves the shipment.
  2. The shopper cancels order 93. Instance B takes order-cancelled, finds no shipment for order 93, and acknowledges the message.
  3. After the time limit, the broker gives order-placed out again. Instance B takes it and saves the shipment as READY.

Order 93 is cancelled, but its parcel is sent.

This can also happen without a crash. If instance A takes order-placed and is slow, instance B can take order-cancelled and finish first.

The broker gave out the two messages in the order they arrived. They were handled in the wrong order for two reasons. Two instances worked on them at the same time, and a message that the broker gives out again comes after messages that arrived later. So with several instances, or when the broker gives a message out again, a consumer cannot count on handling messages in the order they were published.

Handling the messages in any order

The shipping service has to stop order 93 whichever message it handles first. So I change what the shipping service does when it handles order-cancelled and finds no shipment: it saves one anyway, with the status STOPPED:

{ "shopId": "mugshop", "orderId": "93", "status": "STOPPED" }

I add a unique index on the shop ID and the order ID, so there can be only one shipment for each order. The refund record’s index works the same way. The shipping service handles the two messages like this:

  • order-placed: Save a shipment with the status READY. If the unique index rejects it, a shipment for the order already exists, so leave that shipment as it is. Then acknowledge the message.
  • order-cancelled: Save a shipment with the status STOPPED. If the unique index rejects it, a shipment for the order already exists, so set its status to STOPPED. Then acknowledge the message.

Let’s go through the three steps again. In step 2, instance B finds no shipment for order 93, so it saves one as STOPPED. In step 3, instance B tries to save a READY shipment, and the unique index rejects it. The shipment stays STOPPED, and the parcel is not sent.

If the messages are handled in the right order, the shipment is saved as READY and then set to STOPPED, as before. A second copy of order-placed is rejected by the unique index too, so the shipping service never creates two shipments for one order.