Real-Time Updates

The second performance requirement says new ranks show in real time on open pages of the leaderboard. Before choosing how, let’s see how often a page changes.

How often a page changes

Take a page in the middle of the leaderboard. We use the numbers from “Making the Rank Cheaper”:

• About 10 million players are in the middle band, about 20,000 at each rating point.
• One rating change moves a player past about 160,000 others.
• At the peak, 1,000 ratings change a second. Those are spread over the three time controls. If most games are blitz, most of them land on the blitz leaderboard. For a rough upper estimate, we count all 1,000.

Each rating change moves a player across about 160,000 of the 10 million positions in the middle. So a given position in the middle is crossed about $1,000\times 160,000/10,000,000=16$ times a second.

So a page in the middle changes up to about 16 times a second. Nobody can read a page that changes that often.

Agreeing on a delay

So an open page should not show every change as it happens. We agree with the interviewer on a delay: a new rank shows on an open page within 5 seconds. A page a few seconds behind still feels live. A person has time to read it.

This is the target in normal operation. After a failure, some ranks can be wrong for longer, as “Losing the Leaderboard’s Data” explains.

In an interview, this is how you usually handle “real time”. The interviewer says “real time”, and you agree with them on what that means for this feature.

Polling

With a delay agreed, nothing has to reach the client the moment it happens. So I choose polling: each open page sends a request to the leaderboard service every 2 seconds, and shows the response. Polling more often would add requests without showing anything a person could read.

Recall the rule from earlier for choosing between polling, long polling, and a WebSocket. If a change can show up a minute late, polling is enough. If a change must show up within seconds, but changes are rare and only the server sends them, long polling is enough. A WebSocket pays off when changes are frequent and must show up at once.

Here, changes are frequent, but we agreed they do not have to show up at once. Long polling does not help either. A page in the middle changes many times a second, so a long poll would end almost as soon as it starts, and long polling would turn into polling. So polling every 2 seconds, with ordinary requests, is enough.

Let’s check the others anyway:

• A server-sent event stream for each viewer. The leaderboard service would hold an open connection for every viewer and send each one their page every 2 seconds. That is the same work as polling, plus an open connection for every viewer.
• A WebSocket for each viewer. The open page sends almost no data to the server. Moving to another page is an ordinary request. So the communication goes one way, and a WebSocket is more than we need.
• The WebSocket a player already has open to the game service. Players in a game already have one. But then the leaderboard would depend on the game service, and its requests would add load to the instances that run games. That goes against the availability requirement.

The load of polling

At the peak, 100,000 pages are open, and each sends a request every 2 seconds. That is $100,000/2=50,000$ requests a second. Most of them ask for the same pages near the top.

The leaderboard is public, so every viewer of page 1 gets the same response. We said this about the API: two clients that send the same request get the same response. So HTTP caching helps. Each page’s response has this header:

Cache-Control: public, max-age=1

public says any cache may store the response, not only the client that sent the request. So I put a shared cache in front of the gateway: a server that stores the responses and responds to repeated requests from its copy. max-age=1 says the cache may reuse the response for 1 second. So for each page, the cache sends one request to the leaderboard service about once a second, and responds to every other request from its copy. The cache sends the response for almost all of the 50,000 requests.

Let’s check the whole path against the agreed 5 seconds. After a game ends, a new rank waits in three places before an open page shows it:

• The message goes from the game service, through the broker, to the sorted set, and the change reaches the Redis replicas. That usually takes well under a second.
• The cache may keep the old page for up to 1 second.
• The open page sends its next request up to 2 seconds later.

So a new rank shows within about 4 seconds, inside the agreed 5.

The requests for a player’s own rank, with the players around them, have their own URL, and the cache stores those responses too. But each player’s response is different, so most of those requests reach the leaderboard service and Redis. This is why the Redis replicas handle the reads.

There is one more reason to poll. A cache can share a polled response between viewers, but a cache cannot share a stream.