Putting It Together

We can combine horizontal scaling and functional decomposition. The API’s own instances can be duplicated, several of them behind their own load balancer, scoped only to the API now. The registrar’s application scales independently, even if it only ever needs one instance.

graph TB
    subgraph "Clients"
        A[People using the registrar's application]
        B[Other systems using the API, e.g. CourseTracker]
    end

    GW[API Gateway]

    subgraph "Services"
        APP[Registrar's Application]
        subgraph "API"
            LB[Load Balancer]
            API1[API Instance 1]
            API2[API Instance 2]
            LB --> API1
            LB --> API2
        end
    end

    DB[(Database)]

    A --> GW
    B --> GW
    GW --> APP
    GW --> LB
    APP --> DB
    API1 --> DB
    API2 --> DB

Although we separated the API gateway and the load balancer, it is common for load balancing to be included in the job of an API gateway: route by path, then pick an instance within that service.

Every instance in this picture, the registrar’s application and both API instances, still reads and writes the same single database. The API mostly talks to the database and does little computation of its own. We assumed earlier that profiling pointed to the shared server, but in a real deployment, the database would be a likely bottleneck to check first. As traffic keeps growing, the database is what runs out of room next.

Vertical scaling still works on a database, the same as it did on the shared server. Duplicating a database is not as simple as duplicating a stateless server, because a database holds state that has to stay consistent across every copy.

Choosing How to Scale

Vertical scaling, horizontal scaling, and functional decomposition each showed up in this chapter for a different reason. A rule of thumb for choosing among them:

Reach for vertical scaling first. It is initially the cheapest option and requires no architecture change. If that is not enough, reach for horizontal scaling or functional decomposition, depending on the problem.

Reach for horizontal scaling when a single instance cannot hold enough capacity, or cannot be allowed to be a single point of failure. Duplication removes the size ceiling of vertical scaling and further increases the total available resources.

Reach for functional decomposition when two things bundled together grow, change, or need to be owned differently, no matter how many instances either one has. Splitting does not add capacity by itself. It lets each piece scale, change, and fail on its own.