Classic request-response

The monolith CRUD variant is what every service should start as. One application process — Rails, Django, Express, Spring Boot — talking to one Postgres instance. No cache, no queue, no load balancer. The deployment is a single binary or container; the infrastructure is one VM or one managed DB instance.

This is not a compromise. Modern Postgres on a db.r6g.xlarge (4 vCPU, 32 GB RAM) handles 5,000-10,000 transactions per second for a typical CRUD workload with proper indexing. That covers the first two years of most B2B SaaS products, most internal tools, and many consumer products up to 50k DAU. The single-server variant is also the easiest to debug (one log stream), the cheapest to operate (one instance), and the fastest to iterate on (no service boundaries to negotiate).

The mistake is leaving this variant too early. Teams add Redis at 12% DB CPU because "caching is best practice." They add a load balancer at 200 RPM because "we should be stateless." Both are correct principles applied at the wrong time. Every component you add is a component you must monitor, fail over, and debug at 3 AM. Stay here until a specific metric — CPU, connection count, p99 latency — tells you to move.

When to leave. When any of these are true: (a) DB CPU exceeds 50% sustained and growing, (b) peak connection count approaches the DB's max_connections limit, (c) you need horizontal app scaling for availability (multiple AZs), (d) a hot endpoint's latency would benefit measurably from caching. These are concrete, measurable triggers — not vibes.

The cache variant adds Redis in a cache-aside configuration between the app and the primary DB. This is the first scaling move most services make, and it's the right one when: (a) DB CPU is climbing, (b) a subset of reads are hot (80/20 rule), and (c) stale data for 30-60 seconds is acceptable on those endpoints.

Cache-aside contract: app reads Redis; on miss, reads Postgres, writes to Redis with a TTL. On every write to a cached entity, the app invalidates the cache key. TTL is the safety net — even if a write path forgets to invalidate, the stale value expires. Keep TTLs short (30s-5min for most endpoints) rather than optimistically long.

Hit-rate is the metric. Target 80%+ on hot endpoints; alert if it drops below 70%. A declining hit rate means your working set has outgrown cache RAM, your key design is too granular, or your invalidation is too aggressive. Instrument per-endpoint hit rates, not just the aggregate.

The operational cost is real: you now have a Redis cluster to provision, monitor, and fail over. Redis Sentinel or Redis Cluster for HA; memory sizing based on working set + headroom; eviction policy (allkeys-lru for cache-aside). And you inherit the invalidation tax: every write path must know which cache keys to clear. Miss one and users see stale data. This tax is manageable when the cache covers 5-10 entity types; it becomes a full-time job at 50+.

Connection pooling matters more now. With cache in play, each request potentially makes two network calls (Redis + Postgres) instead of one. The app's connection pool to both Redis and Postgres must be sized to handle peak concurrency without queueing.

When the cache hit rate is already 85%+ but the primary's CPU still climbs because of cache misses and the long-tail of uncacheable queries, read replicas are the next move. Two Postgres replicas behind pgBouncer give you 3× the read throughput without changing application code — the connection proxy routes reads to replicas and writes to the primary based on the query.

pgBouncer is the multiplexer. Five app instances × 25 connections each = 125 app-side connections. pgBouncer collapses these into 50 real backend connections to Postgres. Without it, scaling the app tier eventually hits the DB's max_connections wall (default 100 on many managed Postgres instances). pgBouncer's transaction pooling mode gives you the best connection reuse; session pooling if you use prepared statements or SET commands that need session affinity.

Replication lag is the design conversation. Streaming replication in Postgres typically lags by milliseconds under normal load, but can spike to seconds during write bursts or vacuum operations. You need a read-your-writes strategy documented before you ship:

• Write-recency cookie: after a user writes, set a cookie with a timestamp. For reads within 5 seconds of that timestamp, route to primary. After the window, route to replica.
• Cache write-through: on write, populate the cache immediately. The user reads from cache (fresh) while the replica catches up.
• Semi-synchronous replication: one replica must acknowledge before the primary confirms the write. Adds ~1ms latency to writes; guarantees at least one replica is current.

Pick one. Document it. Don't let different teams pick different strategies — you'll get inconsistent behaviour that's impossible to debug.

The operational surface grows: you now manage primary + N replicas + pgBouncer + Redis. Failover playbooks are mandatory. Automated replica promotion on primary failure. Monitoring replication lag with alerting at your SLO threshold.

The graduated hybrid is the natural evolution when some work triggered by a request doesn't need to complete before the user gets a response. The canonical example: user places an order → the API validates, persists to DB, returns 200 with the order ID → then enqueues side-effects: send confirmation email, notify warehouse, update analytics, charge payment asynchronously if pre-authorised.

The contract is simple. The synchronous path does the minimum: validate, persist, respond. Everything else goes to a queue (SQS, RabbitMQ, Redis Streams — pick the one your team knows). Workers consume the queue and process side-effects. If a worker fails, the message retries. The user never sees the failure because they already got their 200.

This variant is not event-driven architecture. You're not building a pub-sub mesh or an event store. You're putting a queue between "thing the user cares about" and "things that happen because of it." The app is still a monolith; the queue is a tactical tool, not an architectural philosophy.

When to use this inside request-response. When your p95 latency budget is 300ms but the full request lifecycle takes 1.2 seconds because of email sending, PDF generation, webhook delivery, or third-party API calls. Move the slow parts to the queue; the synchronous path drops to 50ms.

Failure isolation is the real win. If the email service is down, the order still succeeds. If the analytics pipeline is backed up, users don't notice. Each side-effect retries independently. Compare this to the monolith where a slow SMTP server makes the entire POST /orders endpoint timeout.

The cost is eventual consistency on the side-effects. The user gets their 200 but the confirmation email might take 30 seconds. For most products, that's fine. For anything where the user needs to see the result of the side-effect immediately (e.g., "your payment was charged"), keep that step synchronous and queue only the truly deferrable work.

Modern Postgres on a properly-sized instance handles 5,000-10,000 TPS for a typical CRUD workload. Zero cache, zero replica, zero operational surface beyond one DB. The discipline at this stage is resisting "best practice" that adds components before the numbers justify them.

At this stage, instrument everything: DB CPU, connection count, p99 latency per endpoint, slow query log. These numbers are what tell you when to move to v2. Without them you'll add complexity based on fear instead of data.

Add a load balancer in front of 2+ stateless app instances. Add Redis cache-aside on the top 5-10 hottest endpoints. This alone typically reduces DB CPU by 50-70%.

The LB also gives you multi-AZ deployment for availability. The cache gives you sub-millisecond reads on hot data. The cost: cache invalidation is now your problem, and Redis is another service to operate.

Add 2 read replicas behind pgBouncer. Route reads to replicas, writes to primary. pgBouncer collapses 100+ app connections into 50 backend connections.

Document your read-your-writes strategy before shipping. Replication lag is milliseconds normally but can spike during vacuum or write bursts. Write-recency cookie (pin to primary for 5s after write) is the simplest approach.

When response latency is dominated by side-effects (emails, webhooks, analytics writes), move them to a queue. The synchronous path does validate → persist → respond. Workers handle the rest.

This is the last stop before you leave the request-response pattern entirely. If even the core persist step can't keep up, you're looking at write-heavy or event-driven patterns. If users need push updates, you need real-time. Name the trigger explicitly.

Postgres handles thousands of queries per second but only hundreds of connections. Each connection costs ~10 MB of RAM on the server and a process fork. When you have 10 app instances each opening 25 connections, that's 250 backend connections — above the default max_connections of 100 on most managed Postgres instances.

pgBouncer sits between app and DB, multiplexing N app connections into M backend connections where M << N. In transaction pooling mode, a backend connection is assigned only for the duration of a transaction, then returned to the pool. This means 250 app connections can share 50 backend connections if the average transaction holds a connection for < 20% of the time.

Sizing the pool. Rule of thumb: set pgBouncer's backend pool size to 2-4× the number of CPU cores on the DB instance. A 4-core instance works well with 16-32 backend connections. More connections than cores means context switching and lock contention — you get worse throughput, not better.

App-side pool sizing. Each app instance should open just enough connections to saturate its throughput without queueing. 20-30 per instance is typical. If your app does async I/O (Node.js, Go, async Python), you need far fewer because each connection handles many concurrent queries via pipelining.

Failure mode. When the pool is full and a new request arrives, the request blocks until a connection frees up. If the timeout is too short, the request fails. If it's too long, upstream clients timeout first and retry — making things worse. Set the pool wait timeout to less than your HTTP timeout to fail fast at the right layer.

Every list endpoint must be paginated from day one. The two approaches are offset-based (OFFSET 20 LIMIT 10) and cursor-based (WHERE id > last_seen_id LIMIT 10). Cursor-based wins on every axis that matters in production.

Why offset breaks. OFFSET N forces the DB to scan and discard N rows before returning the page. At page 1000 with 10 items per page, the DB scans 10,000 rows to return 10. As the dataset grows, deep pages get slower linearly. Worse: if rows are inserted or deleted between page fetches, rows shift — users see duplicates or miss items entirely.

Cursor contract. The server returns each page with an opaque cursor (typically the last row's ID, base64-encoded). The client passes it back to get the next page. The server query is WHERE id > decoded_cursor ORDER BY id LIMIT page_size. This is O(1) regardless of depth because the DB seeks to the cursor via the index, not by scanning.

Compound cursors. If sorting by created_at (not unique), the cursor must include both created_at and id: WHERE (created_at, id) > (cursor_ts, cursor_id). This handles ties correctly. The index must match: CREATE INDEX idx ON table(created_at, id).

Client contract. Return a JSON envelope: { "data": [...], "next_cursor": "abc123", "has_more": true }. Clients iterate by passing next_cursor until has_more is false. Never expose raw database IDs as cursors in public APIs — encode them.

Cache integration. Paginated responses are cacheable if the cursor is part of the cache key. First-page results (no cursor) cache well because many users fetch page 1. Deep pages are long-tail and rarely worth caching.

PUT is idempotent by HTTP specification — applying it twice produces the same result. POST is not. But clients retry POSTs: network timeout, 502 from the LB, mobile app backgrounded mid-request. Without idempotency, a retried POST /orders creates two orders. For payments, this is a billing incident.

The Idempotency-Key pattern. Client generates a UUID and sends it in the Idempotency-Key header. Server flow:

1. 1Check Redis for the key. If found, return the cached response immediately — this is a retry.
2. 2If not found, set the key in Redis with status "processing" and a TTL (24-48 hours).
3. 3Process the request. Persist to DB.
4. 4Store the response body in Redis under the same key.
5. 5Return the response.

If the server crashes between steps 2 and 4, the key is in "processing" state. The retry hits step 1, sees "processing," and either waits briefly or returns 409 Conflict. The client retries again after the TTL, and the key is gone — safe to reprocess.

What to store. Cache the full HTTP response (status code + body). The retry must return exactly what the original request returned — same status, same body, same headers. This is critical for clients that parse the response.

Scope. Apply idempotency to mutations that have real consequences: order creation, payment charges, account provisioning. Don't apply it to idempotent-by-design operations (GET, PUT, DELETE) — they don't need it. The overhead is one Redis check per request; apply it where the cost of double-execution exceeds that overhead.

Cache-aside is the default caching strategy for request-response services. The app owns both the read and write paths; the cache is a passive store the app manages explicitly.

Read path: App checks Redis → hit: return → miss: read from Postgres, write to Redis with TTL, return to client. Two round-trips on miss (Redis + Postgres), one on hit (Redis only).

Write path: App writes to Postgres → invalidate (DEL) the Redis key. Do not update the cache on write — that's write-through, which has stronger consistency but higher complexity. Invalidation is simpler: the next read will miss and repopulate.

TTL is your staleness budget. The product requirement "user profiles can be 60 seconds stale" translates directly to TTL=60s. Pick the number from the requirements, not from intuition. Short TTLs (30s-2min) are safe defaults; long TTLs (hours) require aggressive invalidation on every write path.

The invalidation tax. Every write path that touches a cached entity must invalidate. Miss one and users see stale data for up to TTL. Real teams defend with three layers: (1) explicit invalidation in every write handler, (2) short TTLs as a safety net, (3) CDC-based invalidation for write paths the app team didn't write (admin tools, migrations, scripts).

Thundering herd on miss. When a hot key expires, every concurrent reader misses simultaneously and all hit the DB. Defences: (a) single-flight — the first reader loads from DB while others wait on a shared future, (b) probabilistic early refresh — readers randomly refresh before TTL expires, spreading the load, (c) background refresh — a separate process refreshes hot keys before they expire.

Sizing Redis. Estimate working set: number of cached entities × average serialised size. Add 2× headroom for fragmentation and overhead. A 10 GB Redis instance covers most SaaS products' hot data sets. Monitor memory usage and eviction rate — if evictions are happening, you need more memory or shorter TTLs.

The request-response pattern lives or dies on API design. A well-designed API is cacheable, paginated, idempotent, and versioned. A badly designed one creates the problems that push you to "need" more complex patterns.

Resource-oriented URLs. GET /users/123, POST /users, PUT /users/123, DELETE /users/123. Not GET /getUserById?id=123. Resource URLs cache better (the URL is the cache key), are self-documenting, and follow the HTTP method semantics that clients and CDNs understand.

Consistent envelope. Every response: { "data": ..., "meta": { "request_id": "...", "pagination": { ... } } }. Request IDs enable end-to-end tracing. Pagination metadata enables clients to iterate. Consistent envelopes mean one client-side parser, not one per endpoint.

Versioning strategy. URL-path versioning (/v1/users) is the simplest and most visible. Header versioning (Accept: application/vnd.api+json; version=2) is more RESTful but harder to test in a browser. Pick one on day one; changing later is painful.

Error contract. { "error": { "code": "INVALID_EMAIL", "message": "Email format is invalid", "field": "email" } }. Machine-readable code for client logic; human-readable message for debugging. Never leak stack traces or internal IDs in production error responses.

Rate limiting on every endpoint. Even internal APIs. A misbehaving batch job that hammers an unprotected endpoint at 10k RPS will take down the database. Use token bucket per API key or per user ID. Return 429 with Retry-After header. Log rate-limited requests for visibility.

Request validation at the boundary. Validate request bodies (schema, types, ranges) before any business logic. Return 400 with specific field errors. This is the cheapest possible rejection — no DB query, no cache check, no side-effects. Every invalid request that makes it past validation is a wasted round-trip.

The request-response pattern is not forever. But leaving it should be a deliberate decision driven by a specific, measured trigger — not a vague sense that "we've outgrown it." Here are the five triggers and where they point.

1. Push required. The user needs to see updates without refreshing — chat messages, live scores, collaborative editing. Polling is a stopgap (and an expensive one at scale). The answer is WebSocket or Server-Sent Events, which means the real-time delivery pattern.

2. Reads dominate 100:1 or more. When a single write fans out to thousands or millions of reads — video metadata, product catalogs, URL shorteners — the read-heavy pattern with CDN, multi-tier cache, and read replicas is the right shape. Request-response with a Redis cache won't cut it because the cache key space is too large and the hit rate drops.

3. Write bursts exceed primary throughput. If your single Postgres primary can't keep up with write volume — IoT telemetry, event logging, high-frequency trading — you need the write-heavy pattern: write-ahead queues, batch inserts, sharding, or append-only stores.

4. Work takes > 1 second. If a single request requires running an ML model, generating a PDF, calling five third-party APIs, or processing a video, the user can't wait. Return 202 Accepted with a job ID, process asynchronously, and let the client poll or subscribe for completion. This is the long-running-tasks pattern.

5. One trigger → N downstream effects. When a single user action (post a tweet, place an order) must notify feeds, send emails, update analytics, trigger webhooks — and you need each effect to be independently retryable and loosely coupled — you've entered event-driven territory.

The discipline. Before graduating, write down: (a) which trigger you hit, (b) the metric that proves it, (c) which part of the system moves to the new pattern, (d) which parts stay as request-response. Most systems are hybrids — the order API stays request-response while the notification subsystem goes event-driven.