Running Out of Room

Let’s pause and make sure we share an understanding of how the registrar system is architected. Suppose there are two servers: one running the registrar database and the other shared between the registrar API and application. That matches the constraint that the API must run on the registrar’s existing hosting platform, next to the registrar’s system.

graph TB
    subgraph "Clients"
        A[People using the registrar's application]
        B[Other systems using the API, e.g. CourseTracker]
    end

    S[Shared Server<br/>Registrar's App + API]

    DB[(Database)]

    A --> S
    B --> S
    S --> DB

The isolation requirement states that API traffic must not slow down the registrar’s own system. Caching, rate limits, and load shedding help us meet this requirement. But load shedding has a cost of its own: a 503 is downtime for the client that receives it, the same kind that the availability requirement considers.

The availability requirement mandates 99.9 percent uptime. As more developers register for the API and as CourseTracker’s own traffic grows, the API sheds load more often. Every shed request is a request that the API failed to answer, and enough of them push the API below its 99.9 percent target.

The fairness requirement adds to the same problem: a registered client is supposed to get a higher limit than an unknown one, and the registrar’s office wants to raise that limit as more of its own systems register. A higher limit lets more traffic reach the API before shedding starts, which is what fairness wants. But it also lets more traffic reach the shared server, and that is exactly what isolation is supposed to prevent.

So, on the current hardware, isolation and availability now work against each other. Raising one lowers the other. This assumes that the bottleneck is the server shared by the API and application. Suppose profiling points to this shared server, not the database. Now suppose we try vertical scaling to give the server more room. That adds capacity to the server. Assume, for this story, that the upgrade is not enough. Traffic to the API keeps growing, and the shared server keeps running out of room again.