Rate Limiting

Two of the quality attributes are about how many requests the API accepts. Isolation says the people who run the registrar’s system must not notice the API’s traffic, no matter how much of it there is. Fairness says a registered client such as CourseTracker gets more requests per minute than a client the university does not know.

Caching handles most of the traffic but not all of it. A client with a bug can send the same request in a loop with a different search string each time, and none of those hit a cache. A client that becomes popular can send more requests than the registrar’s system was sized for. Either one, left alone, slows down the registrar’s own work. So the API counts requests per client and stops answering a client that sends too many. This is a rate limit.

Who is the client

To count requests per client, the API has to know which client a request came from. For a registered client, the registrar’s office issues an API key, a long random string, when the client registers. The client sends it in every request in the Authorization header:

GET /api/offerings?term=fall-2026 HTTP/1.1
Host: registrar.university.edu
Accept: application/json
Authorization: Bearer 7f3a9c1e5b2d4a8f

Bearer means whoever holds this key is the client it was issued to. The key is a secret. Anyone who has it can send requests as CourseTracker, so it is never put in a URL, where it would be logged and shared, and it is only sent over an encrypted connection, which the constraints already require.

A client that sends no key is an unknown client. The API still sends it a response, because the API is public. It counts the unknown client’s requests by the IP address the requests come from.

The limit

Client	Requests per minute
Registered	600
Unknown	60

I made these numbers up. What the requirements expect is that there are two tiers and the registered one is higher. Sixty per minute is one per second, which is more than a person using an app ever needs and less than a loop can send. Six hundred is enough for CourseTracker’s students searching at the same time.

Over the limit

A client that sends its sixty-first request in a minute gets this:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 23

{
  "error": "Rate limit of 60 requests per minute exceeded. Retry after 23 seconds."
}

429 is a client error. There is something wrong with the request, which is that it came too soon after the others. The Retry-After header says how many seconds until the client may send again. A client that gets a 429 should wait that long and then retry, and it should not retry sooner, because a request sent during the wait is also over the limit and gets another 429.

The response is not cached. It is about this client at this moment, and the next request may be fine.

Queuing excess requests

There is another way to handle a client that sends too many requests. Instead of refusing the request, the server holds it in a queue and sends the response late, so the client gets a slow response instead of a 429. This is called throttling. I did not choose it here because a held request still occupies a connection, and it still reaches the registrar’s database when its turn comes. The client also does not learn that it is over the limit. Refusing the request is cheaper for the server, and the 429 tells the client what happened.

When the limit is not enough

A rate limit protects the registrar from any one client, but not from all clients at once. If enough clients each stay under their limit and the total is still more than the registrar’s system can take, the API has to refuse some requests it would otherwise respond to. It does that with 503 Service Unavailable and a Retry-After header. This is called load shedding. A client treats a 503 the way it treats a 500, by waiting and trying again.

So the API has two ways to refuse a request when there are too many.

Status	When
`429 Too Many Requests`	This client is over its limit
`503 Service Unavailable`	The API is shedding load to protect the registrar