Horizontal Scaling

Chapter 5 named horizontal scaling as the alternative to buying a bigger machine: instead of growing the one server, add more of them. That is the next move here. Instead of upgrading the shared server again, run several instances of it, each identical, each holding the registrar’s own application and the API together, exactly as before.

A load balancer sits in front of the instances. Every request from CourseTracker, or from any other client, still arrives at the same address. The load balancer decides which instance answers it. This architecture continues to meet the technical constraint: every instance still runs on the registrar’s existing hosting platform, next to the registrar’s system, the same as the single server did.

graph TB
    subgraph "Clients"
        A[People using the registrar's application]
        B[Other systems using the API, e.g. CourseTracker]
    end

    LB[Load Balancer]

    subgraph "Server Instances"
        S1[Instance 1<br/>Registrar's App + API]
        S2[Instance 2<br/>Registrar's App + API]
    end

    DB[(Database)]

    A --> LB
    B --> LB
    LB --> S1
    LB --> S2
    S1 --> DB
    S2 --> DB

This works because the API is stateless, a property the REST section already established: every request carries what a server needs to answer it. A request can go to any instance, because no instance is missing something another instance has. Assume, for this example, that the registrar’s own application can be run the same way, so that a whole instance of the shared server, not just the API’s half of it, can be handed to any client the same way.

What still has to be shared

The REST section pointed out that rate limiting needs request counts even though the API is stateless. Duplicating the server now makes those counts a concern. Suppose each instance keeps its own counts. CourseTracker could send 600 requests to one instance and another 600 to a second, and neither instance would consider it over the limit. Adding instances would quietly raise the client’s allowance.

So the instances must coordinate their counts. They could keep them in the shared database, for example, with each request checking and updating the count as one operation so that two instances cannot both claim the last available request.

What duplication buys

Duplication buys two things. Traffic that used to exceed what one server could handle is now spread across several, so there is more room before shedding has to fire. And if one instance crashes or is taken down for maintenance, the load balancer stops sending it requests and sends them to the others instead. The API does not go down with a single machine anymore.

How many instances, and who runs them

Choosing to duplicate (horizontal scaling) is a design decision. The designer’s part is a back-of-the-envelope calculation: given the traffic one instance can handle and the traffic the API actually sees, work out roughly how many instances are needed, then add one more, so that losing an instance still leaves enough to keep up. Adjusting that count as real traffic changes is the job of whoever operates the registrar’s systems, most likely the university’s IT department, using the same load testing chapter 5 ran for HopPress. It is not a number fixed once in a design document.

Most hosting platforms, including the cloud services many universities already use, let an operator change how many instances are running with a configuration change. This is true of vertical scaling too: getting a more powerful instance is a configuration change. Some platforms monitor load themselves and add or remove instances automatically, a feature called autoscaling. If the hosting platform offers this, you can enable it, which itself involves further configuration. You must still decide how many instances to run at minimum and maximum, and what load triggers adding or removing one. The platform does the rest.