Cache Storage

A cache is a role. Something has to play it, and that something has to hold the data somewhere. This section covers the places a cache can hold its data.

Key-value stores

Most application caches are key-value stores. A key-value store holds pairs: a key, which identifies the entry, and a value, which is whatever data goes with it. The only operations are put a value under a key, get the value for a key, and delete a key. There are no tables, no columns, no joins, and no queries beyond lookup by key.

That is enough for a cache, because a cache lookup always asks whether the store has an entry for one key. For HopPress, the key for a post is its post_id and the value is the row from posts, or the rendered page built from it. The key for the published list is a fixed name, since there is only one list.

A key-value store is a data model, not a place. It can be a data structure inside the application or a separate program.

In-memory caches

The speed of an in-memory cache comes from where the data is stored. It keeps its data in memory, not on disk. A memory read is measured in nanoseconds. A disk read is measured in microseconds or milliseconds, depending on the disk. Serving a request from memory avoids the database query and the queue in front of it. That is what saves the two seconds.

An in-process cache is a data structure, usually a map from key to value, in the memory of the running HopPress application. A lookup is a function call. There is no network and no other program. The cache is limited to the memory of the one process, and it disappears when the process restarts. This is the simplest form, and it is the one HopPress starts with.

A cache server is a separate program that does nothing except hold a key-value store in memory. Redis and Memcached are the common ones. The application sends requests to it over the network, so a lookup is a round trip, faster than a database query but slower than an in-process lookup. In return, the cache runs outside the application: it survives an application restart, it can be larger than one process’s memory, and more than one application process can read from it. HopPress does not need this yet.

A database as a cache

A database can play the cache role too. Chapter 4’s materialized view is a cache: a table that holds a precomputed copy of the course history, refreshed from the source tables, so that a read does not run the joins. It is stored on disk, so it is slower than an in-memory cache, and it is maintained by the database rather than the application. It is still a copy kept for speed, with the base tables as the source of truth.

Another common approach is to use a second, faster database as a cache in front of a slower one. The role is the same. Something is a cache when it holds a copy for speed and the copy can be discarded.

The database’s own cache

A relational database has a cache of its own. When it reads a row from disk, it reads the whole page that contains the row, a block of a few kilobytes, and keeps that page in memory in a region called the buffer pool. The next query that needs any row on that page does not read the disk. That is why vertical scaling, which adds more memory, helps the database: a larger buffer pool holds more pages, so fewer queries read the disk.

If the database has its own cache, why does HopPress need one too? The database’s cache saves the disk read, but there is more to a database query than that. The query still arrives over a connection, is parsed and planned, looks up a value in the index, copies the row into a result, and returns over the network. It still waits in the queue, because the database runs a limited number of queries at once. At 500 readers the requests queue up because the database receives more queries than it can run at once.

The application cache removes the database query. On a hit, there is no database connection, no database query, no database queue, and no network trip to the database. The published list and a frequently read post are both served from the application’s own memory.