What GraphQL Gives Up

GraphQL fixed under-fetching and over-fetching on the cart page by changing what an API request looks like. That same change also affects how caching, rate limiting, and gateway routing work.

HTTP caching

Recall that HTTP caching works by storing responses under their URLs. In the registrar’s API, a request for GET /courses/EN.601.226 is the same request whoever sends it. We can cache the response by giving it a Cache-Control header, so that a client or a cache between the client and the server can reuse the response for a set period of time.

In our shop, every GraphQL request is a POST to /graphql. It has one URL, and caches do not normally store responses to a POST. A query for the cart page and a query for the product page have the same URL, so nothing between the client and the API can tell them apart. Suppose your browser requested GET /courses/EN.601.226 last week, and the copy it cached has now expired. So it has to send a new request to the server to get the current data. Between your browser and the server, though, there may be several shared caches, such as proxies and content delivery networks (CDNs). Suppose someone else in your geographic area requested the same resource recently and their response is still cached at one of these shared caches. In a REST API, your request could be served from that cache. You receive the response faster, and the server does not have to process the request again.

This does not mean that caching is impossible for a GraphQL API. It means that you cannot rely on HTTP-level caching and must use application-level caching instead. You can, for example, let the request reach the server, but then respond from a cache rather than reading the data from the database or recomputing the result. In this scenario the cache sits between the database and the API server. It does not save the server from having to process the request, but it does reduce the load on the database and can speed up responses for clients. The other challenge is deciding what to cache and what key to store it under, given how varied GraphQL queries are. This takes careful design, and often more than one caching strategy.

On the client’s side, you can cache responses by query. When the client makes the same query again, it serves the response from its cache instead of sending a new request to the server. The work is in deciding what to cache, how to key the cache entries, and how to invalidate or update them when the underlying data changes. Most GraphQL client libraries have caching built in, but it takes careful configuration to work well.

Rate limiting by request count

We can rate limit by request count in a REST API. In the registrar’s API, we set a limit of 600 requests a minute for a registered client and 60 for an unknown one. Counting requests is a way to estimate the load you are imposing on the server. When we set a limit, we are trying to keep any one client from overwhelming the server. If we set these limits to account for the expected number of clients and the server’s capacity, the clients together should not overwhelm it either. This only works if each request uses about the same amount of server resources. This is typically the case in REST APIs, where each endpoint performs a relatively fixed amount of work.

In a GraphQL API, one request can be very cheap or very expensive, depending on what the query contains. A client might send a query that asks for a list of items, each with a list of sub-items. Each item or sub-item may have nested fields, drawing from other types. Such a request would be equivalent to hundreds or thousands of REST requests. But it gets counted as one request, a single POST to /graphql. Counting a cheap request and an expensive one as one request each does not protect the API from load.

This does not mean rate limiting is impossible for a GraphQL API. There are ways to get a similar effect, but they are more complex and take careful design. They usually involve limiting how expensive one query can be, so that counting requests measures load again. For example, the server can limit how deeply a query may nest. A query past that depth is rejected, and the client has to send several requests to get the same result. You can combine this with assigning a cost to each field and counting the total cost of a query, so that expensive queries are rate limited more strictly than cheap ones.

Gateway routing

An API gateway routes on the path of a request. For example, in the registrar’s system, /api/v1/... goes to the API server and everything else goes to the registrar’s application server. That still works with GraphQL. /graphql goes to the API server and everything else goes to the application server.

The part that stops working is splitting the API itself across services. With REST, a gateway could send /products/... to one service and /orders/... to another. GraphQL has one path. A product query and an order query look the same to the gateway. To send them to different services, the gateway has to parse the query. It gets harder when one query touches several types, because the gateway has to split the query across services and then assemble their responses into one response for the client. Some gateways do this. It adds work to every request and takes careful design.