Asynchronous Processing

Let’s look at how Starbucks handles a line of customers:

1. The cashier takes your order and writes your name on a cup.
2. The cashier puts the cup in a line of cups and takes the next order.
3. A barista takes the next cup from the line and makes the drink.
4. The barista puts the drink on the counter and calls your name.

You do not wait at the register for your coffee. The cashier keeps taking orders while the barista works. In a rush, the line of cups grows. More cashiers do not make drinks faster, because the espresso machine is the limit. For Mugshop, the limit is the shard’s primary.

This is asynchronous processing: the work is accepted now and done later, and whoever asked for it does not wait for it to be done.

Synchronous and asynchronous communication

One program can ask another for work in two ways. In synchronous communication, the caller sends a request and waits for the result before it moves on. In asynchronous communication, the caller hands over the work and moves on without waiting. It learns the result later.

At Starbucks, a cashier who made each drink before taking the next order would work synchronously.

Recall CodeDrill. The application sends each submission to the runner, which runs the code in a container, and the application waits for the results before it responds. That is synchronous. Saving the submission and running it later would be asynchronous.

Accepting an order now and placing it later

Let’s apply this to placing an order. When placeOrder arrives:

1. The commerce service saves the order with a new ID and the status PROCESSING.
2. It sends the order to a program that keeps a list of orders. That program adds the order to the list and responds that it did.
3. The commerce service responds to the storefront with the order’s ID and that the order is processing.
4. Later, another program takes the order from the list and places it.

The shopper does not wait for step 4.

Placing the orders

Another program places the orders. It is part of the commerce service, but it handles no requests. It repeats three steps:

1. Take the next order from the list.
2. Place it: run the transaction on the primary, take the payment through the payment service, and set the order to PLACED.
3. Take the next order.

The transaction comes first. When the mug sells out, the transaction fails, and the payment service is never called. If we took the payment first, every order after the last mug would be charged and then refunded. If the payment is declined, a second transaction puts the mug back.

The order’s status

An order is now saved before it is placed, so it needs a status:

enum OrderStatus {
  PROCESSING
  PLACED
  FAILED
}

PROCESSING is saved when the order is accepted, in step 1. It changes to PLACED after the payment is taken. It changes to FAILED if no mug is left or the payment is declined. Once the mug sells out, the transaction’s conditional update finds no mug left, so every later order fails.

Order 93 from start to end

Here is order 93, from the request to the payment:

sequenceDiagram
    participant S as Storefront
    participant C as Commerce service: handles requests
    participant L as Program with the list
    participant W as Commerce service: places orders
    participant Pay as Payment service
    participant P as Primary
    S->>C: placeOrder
    C->>P: Save order 93: PROCESSING
    C->>L: Order 93
    L->>L: Save order 93 in the list
    L-->>C: Saved
    C-->>S: Order 93, processing
    Note over L: Order 93 waits behind earlier orders
    W->>L: Next order
    L-->>W: Order 93
    W->>P: Run the transaction
    W->>Pay: Take the payment
    W->>P: Set order 93: PLACED

Response time and the rules of placing an order

Recall the problem at noon. Each order is a transaction on the mug’s stock. These transactions run one at a time, at most 100 a second, and 200 orders arrive each second.

With this design, step 1 writes a new document for each order, with no transaction. No two orders write the same data, so they do not wait for each other. Each shopper gets a response within two seconds. Step 4 runs the same transaction as before, at 100 orders a second. The orders it cannot take yet wait in the list. So the commerce service does not hold a request open or refuse an order. The storefront has no reason to send placeOrder again.

Placing an order also has rules. An order must never take the stock below zero, and the order, the stock, and the cart must change together. The transaction keeps these rules. It reduces the stock only if a mug is left. In the same transaction, it fills in the order’s purchase snapshots and total and marks the cart as ordered.