Tutorial 3.2.2

Caching Strategies for High Performance

Chapter 8 · Data Management
~2.5 hours Advanced CDN · Redis · Cache‑Aside · TTL

Overview

Caching is one of the most effective techniques for improving application performance and reducing database load. This tutorial covers the full spectrum of caching strategies: from browser and CDN caching to application‑level in‑memory caches, with practical patterns for invalidation, eviction, and handling high concurrency (cache stampede).

Why this matters: A well‑tuned cache can reduce response times from 200ms to 2ms and handle 10x the traffic without scaling the database. It is a cost‑effective performance multiplier.

1. Cache Hierarchy

Caching exists at multiple layers:

  • Browser (Local) Cache: HTTP headers (Cache-Control, ETag) – fastest, zero network latency.
  • CDN (Edge) Cache: Distributing static assets (images, CSS, JS) globally – reduces latency and origin load.
  • Application (In‑Memory) Cache: Redis/Memcached – stores dynamic data like API responses, user sessions.
  • Database Query Cache: Some databases (e.g., MySQL) cache query results – but often less flexible.

Each layer has a different capacity, speed, and cost profile. Use the appropriate one for your data.

2. Caching Patterns

2.1 Cache‑Aside (Lazy Loading)

The application checks the cache first. If not found (cache miss), it reads from the database, stores the result in the cache, and returns it.

// Pseudocode function getUser(id) { let user = cache.get("user:" + id); if (user == null) { user = db.query("SELECT * FROM users WHERE id = ?", id); cache.set("user:" + id, user, 3600); // TTL 1h } return user; }

Pros: Easy to implement, works for any read. Cons: Cold start penalty, cache misses hit the database.

2.2 Read‑Through

The cache library itself is responsible for loading missing data from the database. The application only interacts with the cache.

2.3 Write‑Through

Writes are performed to the cache, which then synchronously writes to the database. Ensures cache and DB are always consistent.

2.4 Write‑Behind (Write‑Back)

Writes go to the cache, and the cache asynchronously writes to the database. High performance but risk of data loss if the cache fails.

Recommendation: For most web apps, Cache‑Aside is the simplest and most reliable pattern, especially when combined with TTL for automatic invalidation.

3. Invalidation & Time‑to‑Live (TTL)

Cache invalidation is one of the hardest problems in computer science.

  • Time‑To‑Live (TTL): Set an expiry time (e.g., 1 hour). Simple and effective for data that changes infrequently.
  • Event‑driven invalidation: When data is updated, explicitly delete or update the cache key.
  • Versioned keys: Use a version number in the key (e.g., user:123:v2) – when data changes, increment the version.
// Invalidation on update function updateUser(id, newData) { db.update("users", newData, "id = ?", id); cache.del("user:" + id); // Invalidate cache // Optionally: cache.set("user:" + id, newData, 3600); }

4. Distributed Caching (Redis Cluster, Memcached)

When a single cache instance is not enough, you distribute the cache across multiple nodes.

  • Consistent hashing: Distributes keys evenly and minimizes re‑mapping when nodes are added/removed.
  • Replication: Some setups replicate data across nodes for high availability.
  • Sharding: Partition the key space across nodes.

Redis Cluster provides automatic sharding and high availability. Memcached is simpler and faster for pure key‑value caching but lacks persistence.

5. Cache Stampede & Mitigation

A cache stampede (or thundering herd) occurs when a key expires and thousands of concurrent requests all hit the database simultaneously to repopulate the cache.

Mitigation strategies:

  • Mutex locking: Only one thread fetches the data; others wait.
  • Probabilistic early expiry: Refresh the cache before it actually expires (e.g., at 90% of TTL).
  • Stale‑while‑revalidate: Serve stale data while asynchronously refreshing the cache.
// Mutex lock example (Python/Redis) def get_data(key): data = cache.get(key) if data is None: if cache.setnx("lock:" + key, "1"): # Acquire lock data = db.query("...") cache.setex(key, 3600, data) cache.delete("lock:" + key) else: time.sleep(0.01) data = cache.get(key) # Retry return data

Quiz

Question 1

In the Cache‑Aside pattern, what happens on a cache miss?

  • The cache returns an error
  • The application fetches data from the database and populates the cache
  • The database is updated first
  • The request is rejected
Show answer
B. The application fetches data from the database and populates the cache.

Question 2

What is a cache stampede?

  • When a cache is accidentally deleted
  • When multiple concurrent requests attempt to repopulate the same expired cache key
  • When the cache runs out of memory
  • When the cache returns stale data
Show answer
B. When multiple concurrent requests attempt to repopulate the same expired cache key.

Question 3

Which caching pattern asynchronously writes data to the database, providing the highest write throughput?

  • Write‑Through
  • Write‑Behind
  • Cache‑Aside
  • Read‑Through
Show answer
B. Write‑Behind (also called Write‑Back).

Exercises

Exercise 1

A news website uses Redis to cache article content. Articles are updated infrequently. Which invalidation strategy would you choose and why?

Sample answer

Recommendation: Use a TTL (e.g., 5–10 minutes) combined with event‑driven invalidation.

Why: Since updates are rare, TTL ensures freshness without complex invalidation logic. When an article is updated via the CMS, explicitly delete the cache key so the next request fetches the new version immediately.

Exercise 2

Explain the difference between Cache-Control: max-age=3600 and Cache-Control: no-cache.

Sample answer

max-age=3600: The browser can use the cached copy for 3600 seconds (1 hour) without revalidating.

no-cache: The browser must revalidate the resource with the server before using the cached copy (it may still cache it, but must check freshness).

Homework

Homework 1

Compare and contrast Redis and Memcached. When would you choose one over the other? (250–300 words)

Sample answer

Redis: Advanced data structures (lists, sets, hashes, sorted sets), persistence (RDB/AOF), pub/sub, Lua scripting, transactions. Suitable for caching, session storage, leaderboards, queues, and real‑time analytics.

Memcached: Simple, pure key‑value caching. Minimal overhead, multi‑threaded, extremely fast. Lacks persistence and advanced data types.

Choice: Choose Redis when you need data structures, persistence, or durability. Choose Memcached when you need a simple, high‑throughput cache without the overhead of durability and features.

Mini‑Project

High‑Traffic API Gateway Caching

You are building an API gateway that aggregates data from 5 microservices. The external APIs are slow (200ms each). Design a caching layer that reduces the average response time from 1s to under 200ms.

  • Which caching pattern would you use?
  • What TTL would you set for different types of data (user profiles, product catalog, inventory, orders)?
  • How would you handle partial updates (e.g., inventory changes)?
Sample answer

Pattern: Cache‑Aside (Lazy Loading) with Redis.

TTL strategy:

  • User profiles: 10 minutes (rarely change).
  • Product catalog: 1 hour (static).
  • Inventory: 30 seconds (changes frequently).
  • Orders: 1 minute (user‑specific, moderate changes).

Handling updates: When a microservice reports a change (e.g., inventory update), the gateway publishes an event to a message queue. The cache listener invalidates the relevant keys. For inventory, we also reduce TTL to 5 seconds during flash sales.

To prevent cache stampede on high‑traffic items, implement a mutex lock during cache repopulation.

Tutorial Summary

You explored the full caching landscape: from browser and CDN caches to in‑memory distributed stores like Redis. You learned the four main caching patterns (Cache‑Aside, Read‑Through, Write‑Through, Write‑Behind), how to handle invalidation with TTL and versioning, and how to mitigate the cache stampede problem. The exercises and mini‑project gave you practical experience in designing high‑performance caching layers.

Key takeaway: Caching is a cornerstone of scalable architecture. A well‑designed cache reduces latency, cuts infrastructure costs, and dramatically improves the user experience. Always measure your cache hit ratio and adjust TTLs based on data volatility.