How a Load Balancer Works

The domain name in the base URL identifies the server, the same as before. That address used to belong to the single shared server. Now it belongs to the load balancer. The client sees no difference. This is the REST section’s layered system constraint: a client cannot tell whether it is talking to the server directly or to something in between.

The load balancer receives the request, picks one of the running instances, and forwards the request to it. The instance answers as it always did, and the load balancer passes that response back to the client. The client sees one response from one address. It never sees which instance produced it.

A load balancer can be dedicated hardware. More often today, it is software the hosting platform runs alongside the instances it balances. Either way, it checks whether each instance is still answering, and stops sending requests to one that fails the check. One instance crashing does not take the API down.

There is more than one common rule for picking which healthy instance answers a request: cycle through instances in order regardless of load (round robin), send each request to whichever instance has the fewest requests in flight (least connections), or use some other rule. Which rule a load balancer uses is an implementation detail.