Edge caching / CDN-first

The no-CDN topology is where every system starts. Clients resolve your domain to your origin server's IP (or load balancer). Every GET, every static asset, every API call travels from the user's browser across the internet to your data centre and back. If your users are in Sydney and your origin is in us-east-1, every request pays ~200 ms of network latency before your server even starts processing.

Why it works at small scale. For a few thousand daily users, all in one geography, the simplicity is unbeatable. No cache invalidation bugs, no stale content surprises, no CDN bill. Your origin server is the single source of truth for every byte — debugging is trivial because there is exactly one place to look.

Why it breaks. Three forces push you off this topology:

1. 1Latency. Global users experience round-trip times of 150–300 ms to a single-region origin. Multiply by the number of sequential requests to render a page (HTML + CSS + JS + API calls) and the page load exceeds 2 seconds — the threshold where conversion drops measurably.

1. 1Origin load. Every static asset request (logo.png, bundle.js, fonts) hits your application servers. These bytes are identical for every user. Serving them from origin is pure waste — CPU cycles, bandwidth, connection slots consumed for content that never changes.

1. 1Cost. Bandwidth from cloud providers is expensive ($0.05–0.09/GB on AWS). CDN bandwidth is 3–10× cheaper because CDNs negotiate peering agreements at scale. At 100 TB/month, the savings fund the entire CDN bill.

When to stay here. Internal tools, B2B dashboards with <1,000 users in one region, pre-launch MVPs. The moment you have public traffic or global users, add a CDN.

The pull CDN is the default production topology. You point your domain's DNS (CNAME) at the CDN provider. The CDN's PoPs intercept every request. On a cache HIT, the PoP returns the response in <10 ms. On a MISS, the PoP forwards the request to your origin, caches the response according to Cache-Control headers, and returns it to the client.

Setup is minimal. Configure DNS, set Cache-Control headers on your origin responses, and you are live. No asset upload pipeline, no deploy hooks — the CDN populates itself lazily. This is why pull CDN is the right default.

The cold-start problem. Every PoP starts cold. The first request for a given URL at a given PoP always hits origin. With 200+ PoPs worldwide and a long-tail catalogue (10M product pages), most PoP-URL combinations are cold at any point in time. The effective hit rate depends on request concentration: a popular homepage might hit 99% across all PoPs, while a niche product page might be cached in only 3 PoPs.

Cache-Control is the API contract. Your origin controls caching behaviour entirely through HTTP headers. This is elegant — the CDN is a standards-compliant HTTP cache. But it means a misconfigured header (Cache-Control: private on a public page, or no Cache-Control at all) silently breaks caching with no error. Monitor your CDN's HIT/MISS ratio per path — a sudden drop means a header regression.

Vary header traps. Adding Vary: Accept-Encoding is safe (gzip vs brotli, 2-3 variants). Adding Vary: Accept-Language creates one cache entry per language. Adding Vary: Cookie creates one entry per user session — cache is useless. The rule: Vary only on headers that produce meaningfully different responses, and keep the cardinality low.

Cost model. CDN pricing is per-request + per-GB egress, both cheaper than origin. The origin savings from a 90% hit rate typically exceed the CDN cost by 3–5×. Monitor origin offload percentage — it is the single metric that justifies the CDN investment.

Origin shield adds a second caching tier between edge PoPs and your origin. When an edge PoP has a cache miss, instead of going directly to origin, it routes to the designated shield PoP. If the shield has the content, it serves it — edge PoP caches it, done. If the shield also misses, it fetches from origin, caches, and serves all waiting edge PoPs.

Why shield matters: the miss storm. Without a shield, a TTL expiry on a popular URL triggers simultaneous misses from 200+ PoPs, all hitting origin at the same instant. This is a miss storm (also called a cache stampede at the edge). Origin sees a 200× traffic spike for a few seconds every TTL cycle. With a shield, all those PoP misses converge on one shield PoP, which sends exactly one request to origin. The storm is collapsed.

Shield placement. Choose a shield PoP geographically close to your origin — ideally in the same cloud region. This minimises shield-to-origin latency. Most CDN providers let you configure this: Fastly has explicit shield PoP selection, CloudFront has Origin Shield as a checkbox with region selection, Cloudflare uses tiered caching (automatic).

Request coalescing. Advanced CDN configurations combine shield with request coalescing (also called request collapsing). When multiple requests for the same cache key arrive at a PoP within a short window, the PoP sends only one request upstream and fans out the response to all waiting clients. Fastly and Varnish support this natively. Combined with shield, this means even within a single PoP, concurrent misses produce one origin request.

Cost of an extra hop. Shield adds latency on a cold miss: client → edge PoP → shield PoP → origin instead of client → edge PoP → origin. The extra hop is typically 5–30 ms depending on shield placement. For a cold miss that is already paying 50–200 ms of origin processing, this is negligible. The origin protection is worth far more than the latency cost.

When to enable shield. Any time your origin is capacity-constrained or your content has sharp TTL boundaries that cause synchronised expiry. Shield is nearly free (small per-request surcharge) and dramatically smooths origin load.

Edge compute platforms — Cloudflare Workers, Fastly Compute@Edge, Vercel Edge Functions, AWS Lambda@Edge, Deno Deploy — run your code at the CDN's PoPs. The PoP is no longer a passive cache; it executes JavaScript/TypeScript/Wasm in a sandboxed runtime, with access to edge KV stores, and returns computed responses.

Use cases that justify edge compute:

1. 1A/B test bucketing. Hash the user ID at the edge, select a variant, serve the corresponding cached page. No origin round-trip, no client-side flicker.
2. 2Auth token validation. Verify a JWT at the edge and reject unauthenticated requests before they reach origin. Reduces origin attack surface.
3. 3Geo-routing. Inspect the client's country/region (available from CDN headers) and route to the nearest origin or serve region-specific content.
4. 4HTML assembly. Fetch a cached HTML shell and inject personalised fragments from edge KV or a short origin call. This is the programmatic version of shell-and-slot.
5. 5API gateway logic. Rate limiting, request transformation, header injection — lightweight middleware that does not need origin.

Trade-offs to name in an interview:

• Per-request compute cost. Edge compute is billed per invocation + CPU time. At millions of requests per second, this adds up. A cached static response costs $0.01/10K; an edge function invocation costs $0.50–1.00/million on top.
• Limited runtime. Workers have 128 MB memory and 30 s CPU time (often less). No filesystem, no long-running processes. This is a constraint, not a bug — it keeps the edge fast.
• Cold start. V8 isolates (Cloudflare) start in <5 ms. Lambda@Edge containers start in 50–200 ms. Choose your provider based on your cold-start tolerance.
• Debugging. Logs are distributed across 200+ PoPs. You need centralised log aggregation (Logpush, Datadog) to debug production issues.

When edge compute wins over traditional CDN. When you need sub-20 ms response times for logic that varies per request but does not need origin data. When origin round-trip time is the bottleneck and the logic is simple enough to run in a constrained runtime. When you want to move auth, A/B, or geo-routing out of your application servers entirely.

All reads and static assets served by origin servers. Works for small internal apps and pre-launch MVPs. Latency equals client-to-origin RTT; origin bears 100% of traffic.

DNS points to CDN. PoPs cache responses on first miss. Hit rate 80–95% for well-keyed content. Origin load drops by an order of magnitude. Cold-start misses still hit origin for each PoP.

A shield PoP between edge and origin absorbs concurrent misses. Combined with request coalescing, origin load is nearly flat regardless of PoP count. Shield placement should be in the same region as origin.

Edge workers handle auth, A/B bucketing, geo-routing, and HTML assembly. Origin becomes a pure API server. Static content is cached; dynamic content is computed at the edge with sub-10 ms latency.

The cache key is the single most consequential decision in CDN architecture. It determines how many distinct copies the CDN stores and, inversely, how often a request can be served from cache. A bad cache key turns a CDN into an expensive proxy.

Default key: method + URL. A GET to /product/42?color=red stores one entry. A GET to /product/42?color=red&utm_source=google stores a different entry — identical content, separate cache slot. Multiply by every tracking parameter every ad platform appends, and your hit rate collapses.

Rule 1: Strip tracking parameters. Configure the CDN to remove utm_*, fbclid, gclid, msclkid, and similar parameters before computing the cache key. Fastly, Cloudflare, and CloudFront all support query-string whitelisting or blacklisting. Whitelist is safer — explicitly name the parameters that affect the response (color, size, page) and ignore everything else.

Rule 2: Normalise query-param order. ?color=red&size=M and ?size=M&color=red should map to the same cache key. Some CDNs do this automatically; others require edge logic to sort parameters before caching.

Rule 3: Use Vary sparingly. Vary: Accept-Encoding is fine (2-3 variants: gzip, brotli, identity). Vary: Accept-Language is acceptable if you serve genuinely different content per language (5–20 variants). Vary: Cookie is almost always wrong for public content — it creates one cache entry per session, destroying hit rate.

Rule 4: Monitor cardinality. If your CDN reports 50M distinct cache keys but only 2M distinct pages, you have a key explosion problem. The ratio of keys to logical resources should be close to 1:1 (times the number of Vary dimensions).

Cache key for API responses. API GETs can be cached if the response depends only on the URL and a few headers. Example: /api/products?category=shoes with Vary: Accept-Language caches per language. But /api/feed with Authorization header must use Cache-Control: private — the CDN must never serve one user's feed to another.

Interview signal. When a candidate says "put it behind a CDN," ask "what is the cache key?" If they can articulate the key design, strip rules, and cardinality concern, they are operating at a senior level.

TTL (Time To Live) controls how long a cached response is considered fresh. Setting the right TTL per content type is the second most impactful CDN decision after cache key design.

Immutable assets (1 year). Any asset whose URL contains a content hash — app.a1b2c3.js, style.d4e5f6.css — is immutable by definition. If the content changes, the hash changes, and the URL changes. Set Cache-Control: public, max-age=31536000, immutable. The CDN will never revalidate this URL. This is the gold standard for static assets and the reason modern build tools (Webpack, Vite, esbuild) put hashes in filenames.

HTML pages and API responses (short TTL + SWR). These change unpredictably. A 60-second max-age means a price update takes at most 60 seconds to propagate. Adding stale-while-revalidate=300 means the PoP serves stale content instantly for up to 5 minutes while fetching a fresh copy in the background. The user never waits for origin; the cache self-heals within one revalidation cycle.

The SWR flow in detail:

1. 1Request arrives. max-age has expired but SWR window is still open.
2. 2PoP serves the stale response immediately (0 ms added latency).
3. 3PoP sends an asynchronous revalidation request to origin.
4. 4Origin returns fresh response. PoP replaces the cached entry.
5. 5Next request gets the fresh response.

This means staleness is bounded by the SWR window (not the max-age), and the user experience is always fast. SWR is the single best directive for balancing freshness and performance.

Private / personalised content (no cache). Any response that depends on the user's identity — dashboard data, account settings, cart contents — must be Cache-Control: private, no-store. The CDN must never cache it. Use the shell-and-slot pattern to cache the surrounding page while keeping personalised slots private.

s-maxage vs max-age. s-maxage sets the TTL for shared caches (CDN) independently from max-age (browser cache). Example: Cache-Control: public, max-age=0, s-maxage=60 tells the browser to always revalidate but lets the CDN serve cached content for 60 seconds. This is useful when you want the CDN to absorb traffic but the browser to always check freshness.

Conditional requests (ETag / Last-Modified). When a cached response expires, the PoP can send a conditional request to origin: If-None-Match: "etag-value". If the content hasn't changed, origin returns 304 Not Modified (no body), saving bandwidth. ETags are cheap insurance against unnecessary full responses.

Tight TTLs self-heal, but sometimes you need immediate invalidation: a price correction, a security vulnerability in a JS bundle, a legal takedown. CDN invalidation is the escape hatch — use it rarely, but know it cold.

Purge by URL. Call the CDN's purge API with the exact URL: POST /purge { "url": "https://shop.com/product/42" }. Every PoP that has this URL cached invalidates it. The next request triggers a miss, fetches from origin, and re-caches. Simple, but does not scale — if you update 10,000 products, you need 10,000 purge calls.

Surrogate keys (cache tags). When origin serves a response, it attaches a Surrogate-Key header: Surrogate-Key: product-42 category-electronics homepage-featured. The CDN indexes the cached response under all those keys. To invalidate everything related to product 42, you purge by tag: POST /purge { "key": "product-42" }. One call invalidates the product page, the category listing, the homepage feature slot — every URL that carried that tag. Fastly supports this natively with <150 ms global propagation. Cloudflare uses Cache-Tags. CloudFront has limited tag support and relies on Lambda@Edge workarounds.

Versioned URLs. Instead of invalidating, change the URL itself. app.v1.js becomes app.v2.js. The old URL expires naturally per its TTL; the new URL is fetched fresh. This is the standard approach for deploy-time assets (JS, CSS, images). Build tools generate content-hashed filenames automatically. No purge API call needed — the deploy itself is the invalidation.

Soft purge vs hard purge. A hard purge removes the cached entry immediately; the next request is a cold miss. A soft purge marks the entry as stale; the next request triggers a background revalidation (SWR-like behaviour). Soft purge avoids miss storms on popular URLs — users get stale content for a few seconds instead of waiting for origin. Use soft purge as the default; hard purge for security-critical invalidation.

Propagation time matters. Fastly propagates purges in <150 ms globally. CloudFront takes 1–5 minutes. Akamai varies by configuration. If your use case requires <1 s invalidation (security patches, price corrections on high-traffic pages), choose a CDN with fast purge propagation — it is a capability differentiator, not just a speed claim.

Interview tip. When discussing invalidation, name the trade-off: purge is fast but operationally risky (purge the wrong tag and origin gets hammered). Versioned URLs are safe but require a deploy. Tight TTL + SWR is the low-risk default; purge is the emergency lever.

The hardest CDN problem is a page that is 90% shared and 10% personalised. An e-commerce product page has the same product title, images, description, and price for every user — but the cart count, recently-viewed recommendations, and "Hello, Alice" greeting are per-user. Caching the whole page per user destroys hit rate. Serving the whole page from origin wastes the CDN.

Shell-and-slot architecture:

1. 1The shell is the static/shared portion of the page: HTML layout, navigation, product information, footer. It is served by the CDN with a standard TTL (60 s + SWR 300 s). Hit rate: 95%+.

1. 1The slots are the personalised fragments: cart count, user greeting, recommendation carousel, wishlist indicator. They are fetched by client-side JavaScript as small JSON API calls after the shell loads. These API calls use Cache-Control: private, no-store — they never touch the CDN.

1. 1The browser assembles the final page by injecting slot content into placeholder divs in the shell. The user sees the shell instantly (CDN speed), then slots appear within 100–300 ms as API calls complete.

Why this works. The shell is the heavy part — HTML, CSS references, product images, structured data. Serving it from edge saves 80–95% of bandwidth and latency. The slots are tiny (a few hundred bytes of JSON each) and cheap for origin to serve. Total origin load: a handful of small API calls per page view instead of rendering the entire page.

Server-side vs client-side slot filling. Edge compute enables a server-side variant: the edge worker fetches the cached shell, fetches slot data from origin API, assembles the HTML, and returns a complete page. The user gets a fully rendered page from the edge with no client-side JS required. This improves perceived performance and SEO (no layout shift from slot injection). The trade-off is edge compute cost per request.

Cross-reference: read-heavy pattern. Shell-and-slot is the CDN-side complement to the read-heavy pattern's cache-aside strategy. In read-heavy, the application cache (Redis/Memcached) absorbs database reads. In shell-and-slot, the CDN absorbs HTTP reads. A fully optimised read path uses both: CDN serves the shell, browser fetches slots, slots hit the application cache before touching the database. Three layers of caching, each absorbing a different class of traffic.

Pitfall: layout shift. If slots change the page layout when they load (e.g., a recommendation carousel pushes content down), the user experiences Cumulative Layout Shift (CLS), which hurts Core Web Vitals scores. Reserve space in the shell for each slot (fixed-height placeholder) to prevent shift.

When a popular URL's TTL expires, every PoP that had it cached simultaneously discovers the entry is stale. Without protection, 200+ PoPs send concurrent requests to origin. Origin sees a traffic spike of 200× baseline for a few seconds every TTL cycle. This is a miss storm — the edge equivalent of a cache stampede.

How origin shield works. A single PoP is designated as the shield for a given origin. All edge PoPs route their misses to the shield instead of directly to origin. The shield acts as a second-tier cache:

1. 1Edge PoP A has a miss for /product/42.
2. 2Edge PoP A sends the request to Shield PoP (US-Central).
3. 3Shield PoP checks its own cache. If HIT, returns to PoP A.
4. 4If shield also misses, shield sends exactly one request to origin.
5. 5While shield waits for origin, PoP B and PoP C also send misses for /product/42 to shield.
6. 6Shield coalesces all waiting requests. Origin processes one request.
7. 7Shield caches the response and fans it out to PoP A, B, C simultaneously.

Result: origin sees one request instead of 200+. The miss storm is collapsed.

Request coalescing at the shield. This is the key mechanism. When multiple requests for the same cache key arrive at the shield within a short window (typically the time it takes to fetch from origin), the shield holds them in a queue, sends one request upstream, and broadcasts the response to all waiters. Fastly and Varnish call this "request collapsing" and support it natively. Without coalescing, even a shield would forward concurrent misses — just from one PoP instead of 200.

Shield placement strategy. Place the shield PoP in the same cloud region as your origin. Shield-to-origin latency should be <5 ms (same-region) rather than 30–100 ms (cross-continent). This minimises the time window during which other PoPs' misses can pile up.

Multi-origin shield. If you have origins in multiple regions (us-east, eu-west, ap-south), configure separate shields per origin region. Each shield serves the PoPs geographically closest to its origin. This is Cloudflare's "tiered caching" model — a hierarchy of PoPs with configurable topology.

Cost-benefit. Shield adds ~$0.001 per 10K requests and 5–30 ms per cold miss. In exchange, it can reduce origin request volume by 50–90% for content with synchronised TTL expiry. For origins with hard capacity limits (legacy on-prem, rate-limited APIs), shield is not optional — it is a reliability requirement.

Edge compute platforms blur the line between CDN and application server. A Cloudflare Worker or Fastly Compute function runs your code at the PoP, with sub-millisecond cold start (V8 isolates) or <50 ms (Wasm), serving computed responses without an origin round-trip.

Architecture shift. In a traditional CDN setup, the PoP is a dumb cache: it stores responses keyed by URL and replays them. With edge compute, the PoP is a programmable node: it receives a request, runs your handler function, optionally reads from edge KV storage, optionally calls origin for data, and returns a computed response. The PoP is no longer pass-through — it is an application tier.

Where edge compute adds value:

• Auth at the edge. Validate JWTs without an origin round-trip. Reject unauthenticated requests at the PoP. This reduces origin load and attack surface simultaneously. Edge KV stores the JWKS (JSON Web Key Set) for validation.

• A/B testing without flicker. Traditional client-side A/B testing shows the default variant, then flickers to the test variant after JS loads. Edge compute selects the variant before HTML is returned — no flicker, no layout shift, no client-side SDK.

• Geo-personalisation. The CDN injects geo headers (country, city, region). Edge compute uses these to serve region-specific pricing, language, or compliance banners without origin involvement.

• HTML assembly from fragments. The edge worker fetches a cached HTML shell and multiple cached JSON fragments, assembles a complete page, and returns it. Each fragment has its own TTL and cache key. This is ESI (Edge Side Includes) done right — programmatic, testable, debuggable.

• API gateway functions. Rate limiting, request validation, header transformation, CORS handling — lightweight middleware that runs at the edge instead of at origin.

Constraints to name in an interview:

1. 1Memory limit. Workers: 128 MB. Lambda@Edge: 128–512 MB. You cannot load a machine learning model or process large files.
2. 2CPU time. Workers: 10–50 ms CPU time per request (plan-dependent). Long computations must happen at origin.
3. 3No persistent connections. Edge runtimes are request-scoped. You cannot maintain WebSocket connections or database connection pools. Each request opens connections fresh (or uses connection pooling services like Hyperdrive).
4. 4Distributed state. Edge KV stores are eventually consistent with ~60 s propagation. If you need strong consistency, you need origin.

Cost model. Workers: $0.50 per million requests + $12.50 per million ms CPU time. At 100M requests/month, that is $50 in invocations alone — cheap for auth routing, expensive for heavy computation. The decision framework: if the logic is <5 ms CPU and benefits from geographic proximity, run it at the edge. Otherwise, keep it at origin.