Caching Part 1: Where to Cache and Why

Why caches work so well

Caching works because of how people actually use apps. Access is very uneven:

• A small number of products, posts or videos get most of the views. Today's trending video gets millions of views; a video from 2014 gets a few.
• The same user reloads the same pages.
• Popular data is requested again and again within seconds.

This is often called the 80/20 rule: roughly 20% of the data gets 80% of the requests. It is a rough rule of thumb, not an exact law. Keep that 20% in a fast cache and you handle most traffic cheaply.

Hit ratio: the number that matters

Say a cache hit takes 1 ms (one millisecond, a thousandth of a second) and a database query takes 50 ms. The average latency is hits × 1 ms plus misses × 50 ms:

Going from 95% to 99% cuts database load by five times. Small hit-ratio improvements have big effects.

Where can you cache?

Caches exist at every layer of a system, from the user's device to the database:

1. Client cache. The browser keeps files according to HTTP headers (post 14). Mobile apps keep data in memory or on disk.

• ✅ The fastest possible: no network at all.
• ❌ You can't easily clear it from the server side.

2. CDN cache. Covered in post 14. Best for static files and public content.

3. Reverse-proxy cache. A reverse proxy is a server in front of your app that passes requests on. NGINX or Varnish in this role can cache whole HTTP responses. It's good for public pages that are expensive to build.

4. In-process (local) cache. A map or LRU cache inside your application's memory. LRU (least recently used) means that when the cache is full, it removes the item that was used longest ago.

• ✅ Extremely fast (nanoseconds to microseconds), with no network hop.
• ❌ Each server has its own copy. With 20 servers you have 20 caches, which can disagree and each warm up separately.
• ❌ Lost on every restart or deploy, and uses the app's memory.
• Good for: configuration, feature flags, small lookup tables, and very hot keys.

5. Distributed cache (Redis, Memcached). A separate cache server, or cluster (a group of servers that work as one), shared by all app servers.

• ✅ One shared copy, so all servers see the same cached data.
• ✅ Survives app restarts, and can hold much more data.
• ❌ A network hop, typically well under a millisecond inside a data centre.
• ❌ Another system to run, monitor and scale.
• This is what people usually mean by "add a cache".

6. Database cache. Databases keep recently used data pages in memory (called shared buffers in PostgreSQL and the buffer pool in MySQL). That's why a query is often faster the second time. You get this for free, but it can't save you from running expensive queries at huge volume.

What should you cache?

Good candidates:

• Read-heavy data that changes rarely: product details, user profiles, article pages, settings.
• Expensive results: complex queries, aggregated counts ("1.2M views"), recommendation lists, rendered HTML.
• Results from slow or rate-limited external APIs: currency exchange rates, weather, maps.
• Session data (post 7: the record of who is logged in), for fast lookups on every request.

Poor candidates:

• Data that changes on almost every read.
• Data that must be perfectly up to date: an account balance at the moment of payment, or remaining stock during checkout. Read those from the source, or cache very carefully.
• Huge objects that are rarely re-read.

Cache the object, not just the query

There are two common styles:

• Query-level caching. The key is the SQL query (or a hash of it), and the value is the result. It's easy to add, but hard to invalidate. When a product changes, which cached queries contain it?
• Object-level caching. The key is the thing itself, like product:991 or user:42:profile. The value is the assembled object. When product 991 changes, you know exactly which key to update or delete.

Object-level caching is usually cleaner and easier to keep correct.

Cache keys and TTLs

A cache works like a big dictionary. Each item has a key (its name) and a value (the data). Keys should be clear and structured:

product:991user:42:profileuser:42:orders:page:1feed:home:region:IN

TTL (time to live) is how long an entry stays before it expires and is removed:

• short (seconds) for fast-changing data like prices or scores,
• long (hours or days) for data that rarely changes,
• a TTL on everything as a safety net, so mistakes eventually fix themselves.

Redis vs Memcached

Both are in-memory key-value stores (databases that keep data in RAM and look it up by key) used as distributed caches.

Most new projects choose Redis (or a compatible alternative like Valkey) because it does much more than caching: rate limiters (which refuse clients that send too many requests), leaderboards with sorted sets, and simple queues. Memcached is still great for huge, simple cache fleets.

A simple cache in code (cache-aside)

Cache-aside means the application code manages the cache itself: it checks the cache first, and on a miss it reads the database and then fills the cache. This is the most common pattern. We'll cover all the strategies in Part 2.

async function getProduct(id) {  const key = `product:${id}`;  // 1. Try the cache  const cached = await redis.get(key);  if (cached) return JSON.parse(cached);          // hit  // 2. Miss: load from the database  const product = await db.products.findById(id);  // 3. Save to cache with a TTL (e.g., 10 minutes)  await redis.set(key, JSON.stringify(product), "EX", 600);  return product;}

Measure it

Always monitor:

• hit ratio (overall and for important key groups),
• latency (cache vs database),
• memory usage and evictions (an eviction is when the cache removes an item to make space; many evictions mean the cache is too small),
• database load before and after.

A cache with a 20% hit ratio adds complexity without much benefit. Find out why before adding more.