Unit 5.1 · Tutorial 2

Scalability & Caching Strategies

Chapter 12 · System Design & Deployment
~2.5 hours Advanced Scaling · Caching · CDN · Load Balancing

Overview

As your user base grows, your system must scale to handle increased load. This tutorial covers horizontal and vertical scaling, database scaling techniques, caching strategies, and CDN integration. You'll learn to build systems that can grow with demand.

Why this matters: Systems that can't scale fail under pressure. Proactive design for scalability ensures your application remains responsive and available as traffic increases.

1. Scaling Strategies

Vertical Scaling (Scale‑up)

Adding more resources (CPU, RAM) to a single machine. Simple but limited by hardware capacity and expensive.

Horizontal Scaling (Scale‑out)

Adding more machines to distribute the load. Requires statelessness and load balancing. More flexible and cost‑effective.

// Load balancer configuration example (Nginx) upstream my_app { least_conn; server 10.0.0.1:3000; server 10.0.0.2:3000; server 10.0.0.3:3000; } server { listen 80; location / { proxy_pass http://my_app; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } }
Best practice: Design your application to be stateless so it can be scaled horizontally. Store session data in Redis or a database.

2. Database Scaling

Replication

Create read replicas to offload read queries. Write queries still go to the primary.

Sharding (Partitioning)

Split data across multiple databases based on a shard key (e.g., user_id). Increases write throughput but complicates queries.

Connection Pooling

Reuse database connections to reduce overhead. Use tools like PgBouncer.

// Connection pooling with Prisma const prisma = new PrismaClient({ datasources: { db: { url: env.DATABASE_URL } }, // pooling handled automatically });

3. Caching Fundamentals

Caching stores frequently accessed data in fast storage to reduce latency and database load.

  • In‑memory cache: Redis, Memcached – extremely fast.
  • Browser cache: HTTP cache headers (Cache‑Control).
  • CDN cache: Edge‑cached static assets.
  • Database query cache: Caching query results (e.g., MySQL query cache).
// Redis caching example (Node.js) import Redis from 'ioredis'; const redis = new Redis(); async function getProducts() { const cached = await redis.get('products'); if (cached) return JSON.parse(cached); const products = await Product.findMany(); await redis.setex('products', 3600, JSON.stringify(products)); return products; }

4. Caching Patterns

Cache‑Aside (Lazy Loading)

Application checks cache first; on miss, fetches from DB and stores it. Most common pattern.

Write‑Through

Writes go to cache first, then synchronously to DB. Ensures consistency but adds latency.

Write‑Behind (Write‑Back)

Writes go to cache, then asynchronously to DB. High throughput, risk of data loss.

// Cache‑Aside implementation async function getUser(id) { const user = await redis.get(`user:${id}`); if (user) return JSON.parse(user); const dbUser = await User.findById(id); await redis.setex(`user:${id}`, 600, JSON.stringify(dbUser)); return dbUser; }

5. Content Delivery Networks (CDN)

CDNs distribute your static assets (images, CSS, JavaScript) across edge servers worldwide, reducing latency for users.

  • Popular CDNs: Cloudflare, Akamai, Fastly, AWS CloudFront.
  • Benefits: Faster load times, reduced origin load, DDoS protection.
  • Use: Host images, JS, CSS, and even dynamic content (with edge workers).
// Using a CDN with Cloudflare Workers (edge caching) export default { async fetch(request) { const url = new URL(request.url); if (url.pathname.startsWith('/static/')) { const cache = caches.default; const cached = await cache.match(request); if (cached) return cached; const response = await fetch(request); const cacheResponse = new Response(response.body, response); cacheResponse.headers.set('Cache-Control', 'public, max-age=31536000, immutable'); await cache.put(request, cacheResponse.clone()); return cacheResponse; } return fetch(request); } };

Quiz

Question 1

What is the difference between vertical and horizontal scaling?

  • Vertical adds more machines; horizontal adds more resources to one machine.
  • Vertical adds more resources to one machine; horizontal adds more machines.
  • They are the same.
  • Vertical is for databases only.
Show answer
B. Vertical adds more resources to one machine; horizontal adds more machines.

Question 2

Which caching pattern writes data to the cache and then asynchronously to the database?

  • Cache‑Aside
  • Write‑Through
  • Write‑Behind
  • Read‑Through
Show answer
C. Write‑Behind.

Question 3

What is the primary benefit of using a CDN?

  • Reduced database load
  • Lower latency for users globally
  • Better SEO
  • Automatic code updates
Show answer
B. Lower latency for users globally.

Exercises

Exercise 1

Design a caching strategy for a product catalogue API that receives 10,000 requests per second. Describe where you would place caches and what TTL you would use.

Sample answer
  • Cache layers: CDN for product images, Redis for product details (TTL 1 hour), browser cache for static assets.
  • Invalidation: On product update, clear Redis cache and purge CDN.
  • Redis: Use cache‑aside pattern with keys like `product:{id}`.

Exercise 2

Explain how you would scale a monolithic application to handle 10x more traffic. Consider code, database, and infrastructure changes.

Sample answer
  • Infrastructure: Use a load balancer and run multiple instances of the monolith.
  • Database: Add read replicas, use caching (Redis) for frequent queries.
  • Code: Ensure statelessness (move sessions to Redis), optimise database queries, add indexes.

Homework

Homework 1

Design a scalable architecture for a social media feed. Include caching, database scaling, and load balancing. Draw a high‑level diagram and write a description.

Sample outline
  • Load balancer: Distributes traffic across web servers.
  • Web servers: Stateless instances of the API.
  • Cache: Redis for user feeds (pre‑computed timelines).
  • Database: PostgreSQL with read replicas for feed queries.
  • CDN: For images and static assets.
  • Diagram: Include arrows showing data flow.

Mini‑Project

Scalable E‑Commerce Architecture

Design a scalable e‑commerce architecture from scratch. Include:

  • Load balancer and horizontal scaling for the API.
  • Database replication and sharding strategy.
  • Redis caching for product catalogue and user sessions.
  • CDN for static assets and product images.
  • A diagram (Level 2 C4) showing containers and interactions.
Sample outline
  • Load Balancer: Nginx/HAProxy distributing traffic to Node.js API instances.
  • API: Stateless Node.js with Express, running in Docker containers.
  • Cache: Redis for product data, session storage, and rate limiting.
  • Database: PostgreSQL with primary and multiple replicas; shard by user_id.
  • CDN: Cloudflare for images, CSS, JS.
  • Diagram: Draw containers: CDN → Load Balancer → API → Redis + DB.

Tutorial Summary

You learned how to scale applications using vertical and horizontal scaling, database replication and sharding, caching strategies, and CDN integration. These techniques are essential for building systems that can handle growth and maintain performance.

Key takeaway: Scale by design. Plan for growth from the beginning by using stateless architectures, caching, and distributed data patterns.