Caching: The Hard Problem Is Invalidation

"There are only two hard things in computer science: cache invalidation and naming things" — Phil Karlton. The cache itself is the easy part; the hard part is knowing when what's in it is wrong.

The patterns

Read-through (cache-aside) — the app checks the cache first, falls back to the database on miss, writes back. The most common pattern; the flaw is the cold-start miss and the stampede when a hot key expires.

def get_user(id):
    user = cache.get(f"user:{id}")
    if user is None:
        user = db.get(id)
        cache.set(f"user:{id}", user, ttl=300)
    return user

Write-through — every write goes to cache and DB. Keeps the cache warm; pays write latency on every write.

Write-back — writes go to cache, the DB update happens async. Fastest writes; the data loss risk is real (cache dies before the write flushes).

The stampede problem

Ten thousand concurrent requests for an expired hot key: every one misses, every one hits the DB, the DB falls over. Solutions:

• Request coalescing — one in-flight fetch per key; the rest wait.
• Stale-while-revalidate — serve the stale value while one request refreshes in the background.
• TTL jitter — ttl = 300 + random(0, 60) spreads expiry so keys don't all die at once.

Cache keys are a schema

user:{id}, feed:{user_id}:page:{n} — a naming convention that encodes what varies and what doesn't. The mistake is a key that doesn't include everything that could make the value stale: cache feed:{user_id} when the feed depends on a filter, and different users' filtered feeds collide.

::exercise(stampede-name)

Field notes

• The best cache is often a CDN at the edge — cache Cache-Control headers and let the CDN absorb the read load before it ever reaches you.
• "Cache invalidation" isn't a failure mode to prevent; it's a design constraint to accept. Choose the staleness bound the product can live with (300s? 60s? 0?) and pick a TTL that enforces it.