Caching

Look at what the database is being asked to do under load. Five hundred readers request the list of published posts. It is the same list for all of them. Most of them then open one of the same few recent posts. For each request, HopPress runs the query, the database finds the rows, and the rows come back. The answer is the same as it was for the previous reader, and the database computed it again.

In Chapter 4, we denormalized HopPress’s schema to make reads cheaper. We kept state on posts so the database would not derive it from post_state_changes. For CourseTracker, we stored the course history as a materialized view so the database would not run five joins per visit.

A cache is a component whose job is to hold a temporary copy of data so that requests for that data can be served faster than by going to where the data lives. The word names a role in the design, not a particular technology.

Three things follow from the definition:

• A cache holds a copy. The original is somewhere else, and that somewhere else is the source of truth. For HopPress, the source of truth is the database.
• The copy is temporary. It can be thrown away at any time, and the system still works, because the source of truth still has the data.
• The copy lets the system serve requests faster than reading from the source. If it did not, there would be no reason to keep it.

Caches show up at every level of a computer system, from the processor to the browser. This chapter is about one of them: a cache the application keeps in front of its database.

Hit, miss, and hit ratio

A cache typically keeps copies in storage that is faster, and therefore scarcer, than the source of truth. HopPress’s database holds every post ever written, tens of thousands over many years, each with a body of thousands of characters. The cache cannot hold all of that, and it does not need to, because most of those posts are never read again. Most readers open the same few recent posts. So the cache holds the data that is asked for often, and the rest stays in the database.

The cache starts empty. Nothing is in it until a request asks for something, the application reads it from the database, and puts a copy in the cache. So at any moment the cache holds the data that has been asked for recently.

A request asks the cache for a post. If a copy is there, because someone asked for that post before, that is a hit. The request is served from cache rather than the database. If no copy is there, because nobody has asked for that post yet or because its copy was dropped, that is a miss. The request goes to the database as it would without a cache, and the application puts a copy in the cache before returning the result.

Count requests over some period. The portion that hit is the hit ratio. If 900 of 1,000 requests hit, the ratio is 0.9, and the database received 100 queries instead of 1,000.

The hit ratio depends on the data and the requests. A cache is worth having if the hit ratio is high enough that the queries it saves the database justify the memory it takes. We can measure the hit ratio in production and decide whether to keep the cache.