The Complete System Design Architecture

Layer 1: Clients

What: web apps, mobile apps, and other companies' systems (partners) that use your APIs. An API (application programming interface) is a set of rules that lets one program ask another program for data or actions.

Key ideas:

• Never trust the client. A user can change anything that runs on their own device. So validate (check the data is correct) and authorise (check the user is allowed) everything on the server (post 52).
• Clients can help. They can keep a local cache (a saved copy of data). They can retry with backoff and jitter (wait longer after each try, plus a random extra time, post 41). They can keep an offline queue (actions saved until the network returns). They can also prefetch (load early) the next page.
• Mobile apps live for years, so APIs must stay backward compatible (post 34).
• Real-time features use WebSockets or SSE (post 33). A WebSocket is a connection that stays open so both sides can send messages at any time. SSE (server-sent events) is a connection where only the server sends messages. Notifications that arrive when the app is closed use push (post 62).

Posts: 1, 3, 30–34, 41, 62.

Layer 2: Edge (DNS, CDN, WAF)

What: the first things a request touches, often in a city near the user.

• DNS (the domain name system) turns names like example.com into IP addresses. GeoDNS gives each user the address of the nearest region. Anycast uses one address that many places share, so the network sends the user to the closest place. Use two DNS providers for critical sites, so one failure does not stop everything (posts 1 and 11).
• The CDN (content delivery network) is a group of servers around the world. It caches static files, images, video segments and public pages at the edge (close to users). This cuts latency (the waiting time) and the load on your origin servers (post 14).
• A WAF (web application firewall) blocks common attacks, such as bad input in requests. DDoS protection stops a flood of fake traffic sent to overload you. Both work before the traffic reaches you (posts 14 and 52).
• TLS (the protocol that encrypts HTTPS traffic) often ends here or at the load balancer. That means the edge decrypts the traffic (post 3).

Rule of thumb: anything public and cacheable should be served from the edge.

Layer 3: Entry and traffic management

What: components that receive requests and send them to the right place.

• Load balancers spread traffic over many servers. They run health checks (they test if a server is working) and support zero-downtime deploys (new versions with no break in service). L4 balancers route by IP address and port. L7 balancers read the HTTP request and can route by URL or header. Keep them redundant across zones (post 12).
• The API gateway is one front door for all API calls. It checks authentication, applies rate limiting (a cap on requests per client), routes calls to services and checks API keys. A BFF (backend for frontend) is a gateway made for one kind of client, such as the mobile app (posts 13, 43 and 61).
• WebSocket gateways hold long-lived connections for chat and live updates. A presence and session registry is a list that says which user is online and which gateway holds their connection (posts 33 and 63).

Rule of thumb: put generic concerns (auth, rate limits, routing) here, and keep business logic in services.

Layer 4: Application services

What: where business logic runs: orders, payments, users, feeds and so on.

• Stateless services (post 7) keep no user data in their own memory between requests. So any instance (running copy) can handle any request, and you scale out by adding instances.
• Monolith vs microservices (post 53). A monolith is one application. Microservices are many small applications. Start with a (modular) monolith, which is one application with clear inner parts. Split into services when your teams and scale need it.
• Communication: use REST or GraphQL to talk to clients. REST uses URLs and HTTP methods. GraphQL lets the client choose exactly which fields it wants. Use gRPC between services. gRPC is a fast call format that uses Protobuf (a compact binary data format) (posts 30–32). Service discovery lets services find each other's addresses (post 54).
• Resilience on every call. Resilience means surviving failures. Use timeouts (stop waiting after a set time) and a few retries with jitter. Use circuit breakers (stop calling a service that keeps failing), bulkheads (separate resource pools, so one failure cannot use everything) and fallbacks (a simpler backup answer) (posts 41–42).
• Runs on: containers (post 55). A container packs an app with everything it needs to run. Kubernetes (post 56) starts, stops and restarts the containers for you. They run across 3 availability zones, which are separate data centres in one region (posts 40 and 57).

Layer 5: Cache

What: fast, in-memory copies of hot data (post 15).

• Redis cluster. Redis is a fast database that keeps data in memory. A cluster is several Redis machines working together. Use it for: object cache (product, profile), sessions (login state, post 49), counters, rate-limit buckets (post 43), feed lists (post 64), leaderboards and presence (who is online).
• Patterns: cache-aside means the app checks the cache first, and on a miss it reads the database and fills the cache. Delete the cached item when the data changes. Put a TTL (time to live, an expiry time) on everything (post 16).
• Protect against: stampedes (many requests miss the cache together), avalanches (many items expire together; random TTL jitter helps), penetration (requests for data that does not exist, so the cache never helps) and hot keys (one key with far more traffic than the rest) (post 16).
• Always plan for "the cache is down" (post 16).

Layer 6: Databases (the source of truth)

What: durable, authoritative data.

• Default: PostgreSQL/MySQL (relational databases) with a primary + replicas across zones. The primary takes the writes. Replicas are copies that serve reads and take over if the primary fails (posts 5, 20 and 24).
• Scale path: indexes (lookup tables that speed up queries) → caching → a bigger machine → read replicas → functional split (different tables on different databases) → sharding (splitting one table across machines) with a good shard key. The shard key is the field that decides which machine holds a row (posts 22 and 24–26).
• Special workloads: wide-column databases (Cassandra/ScyllaDB) suit huge streams of writes, like chat messages. Key-value databases (DynamoDB) suit simple lookups by key at massive scale (posts 20 and 29).
• Correctness: a transaction is a group of changes that all succeed or all fail. Isolation levels say how much running transactions can see of each other. Unique constraints stop duplicate values. Idempotency keys make repeated requests safe (posts 21 and 34).
• Consistency: strong consistency (everyone sees the latest data) for money and uniqueness. Eventual consistency (copies may differ for a short time) for feeds and counts (posts 27–28).
• Backups + PITR in another region. PITR (point-in-time recovery) lets you restore the database to an exact moment, such as just before a mistake. Replication is not a backup, because replicas copy mistakes too (post 45).

Layer 7: Object storage

What: a service that stores files as objects, such as Amazon S3. It holds images, videos, documents, exports and backups (post 17).

• Pre-signed URLs: a pre-signed URL is a link with a built-in, time-limited permission. Clients upload and download directly, without going through app servers.
• The CDN in front for fast delivery.
• Lifecycle rules move old data to cheaper storage classes automatically.
• Private by default, encrypted, and versioned (old copies are kept) for important data.

Layer 8: Search

What: full-text search (finding words inside text), filters and facets (counts per category), and sometimes vector search (finding items with similar meaning) (post 19).

• Elasticsearch/OpenSearch, or PostgreSQL full-text search for smaller needs.
• Kept in sync via CDC or events. CDC (change data capture) reads the database's changes and sends them on. Search is never the source of truth, because it can always be rebuilt from the database.

Layer 9: Asynchronous processing

What: everything the user doesn't need to wait for.

• Queues (SQS, RabbitMQ) hold jobs until a worker takes them: emails, image processing, reports (posts 18 and 38).
• An event log (Kafka) stores events in order so many services can read them. It is used for event streaming between services, CDC and analytics pipelines (posts 35–36 and 39).
• The outbox pattern + CDC reliably connect database changes to events. The service saves the event in an "outbox" table in the same transaction as the data. A separate step then publishes it (post 37).
• Workers are programs that take jobs from a queue. They use idempotent processing (doing a job twice is safe) and retries with backoff (wait longer after each try). Jobs that keep failing go to a DLQ, a dead-letter queue, which holds them for a person to check (posts 37–38).
• Workflows and sagas handle business processes with many steps. If one step fails, a saga runs compensating actions, which undo the earlier steps (post 39).

Rule of thumb: save the essential state synchronously, and do the rest asynchronously.

Layer 10: Data and analytics

What: reporting, dashboards, machine learning and business intelligence.

• Data flows from databases and events (via CDC or Kafka) into a warehouse or lake. A warehouse is a database built for large reports (BigQuery, Snowflake, ClickHouse). A lake stores raw data files cheaply.
• Stream processing (Flink, Kafka Streams) works on events as they arrive. It is used for real-time metrics, fraud detection and alerts.
• Keep heavy analytics away from the production database (post 5).

Layer 11: External services

What: payment gateways, SMS, email, push notifications, maps and identity providers.

• Treat them as unreliable. Use timeouts. Retry only for transient errors (short problems that go away). Add circuit breakers and backup providers for critical channels (posts 41–42 and 62).
• Respect their rate limits with token buckets. A token bucket gives you a few "tokens" per second, and each call spends one (post 43).
• Verify webhooks with signatures, and handle them idempotently. A webhook is a call that the other service sends to your server when something happens. A signature proves it is real (post 34).

Layer 12: Cross-cutting concerns

These apply to every layer.

Observability (posts 46–48):

• structured logs (records in a fixed format), metrics (numbers over time) and distributed traces (the path of one request across services). Use the Golden Signals (latency, traffic, errors, saturation), RED (rate, errors, duration) and USE (utilisation, saturation, errors) as checklists. OpenTelemetry is a standard set of tools for collecting all of this,
• SLOs and error budgets for key user journeys. An SLO is a reliability goal. An error budget is how much failure the goal still allows,
• symptom-based alerts (alert on what users feel), runbooks (written steps for fixing a problem), on-call (someone is ready to respond) and blameless postmortems (outage reports that look for system causes, not people to blame).

Security (posts 49–52):

• AuthN (authentication: proving who you are, with sessions, JWT, OAuth/OIDC, MFA or passkeys) and AuthZ (authorisation: what you may do, with per-object checks and RBAC/ReBAC, which grant access by role or by relationship),
• TLS everywhere, mTLS between services (both sides prove who they are), KMS and envelope encryption (a key service that protects the keys which encrypt your data), and a secrets manager (a safe place for passwords and keys),
• OWASP basics. OWASP is a group that lists the most common web security problems. The basics are parameterised queries, patched dependencies, SSRF protections, security headers and logging.

Delivery (post 58):

• CI (continuous integration): build, test and scan every change. CD (continuous delivery): release with canary or blue-green rollouts and automatic rollback. A canary sends the new version to a few users first. Blue-green runs the old and new versions side by side and switches traffic.
• Feature flags (switches that turn features on or off), expand-and-contract database migrations (first add the new column, then move the code, then remove the old column), and GitOps (the live system is set by files in Git).

Platform (posts 55–57):

• cloud regions and AZs (a region is a place in the world; an AZ is one data centre inside it), VPC networking (your own private network in the cloud) and infrastructure as code (servers and networks created from files, not by hand),
• Kubernetes (or a simpler managed platform), autoscaling (adding or removing servers as load changes), and cost management.

Resilience and DR (posts 40, 44 and 45):

• no single points of failure (one part that stops everything if it breaks), multi-AZ by default, multi-region for critical systems,
• load shedding (refuse some requests on purpose during overload) and graceful degradation (turn off less important features), backups, PITR and DR drills (DR is disaster recovery, and a drill is a practice run), and chaos testing (breaking things on purpose to see how the system reacts).

How requests flow through the architecture

Flow 1: A read ("show product page")

Key ideas: CDN for static content, cache-aside for data, replicas for reads, and fallbacks for optional parts.

Flow 2: A write ("place order")

Key ideas: transactions for correctness, idempotency for safe retries, the outbox for reliable events, and async fan-out for everything else. Multi-step flows can be an orchestrated saga.

Flow 3: Real time ("chat message")

Key ideas: store first, then deliver, with per-conversation ordering and sync on reconnect.

Flow 4: Background job ("export my data")

How the architecture grows: from 1 user to millions

Remember: move to the next stage when measurements show you need it, not because a big company does it.

Decision cheat sheet

Mapping non-functional requirements to architecture

Common anti-patterns in the big picture

• ❌ Everything synchronous: one slow dependency makes every page slow.
• ❌ A shared database between many services: this is a "distributed monolith" (post 53). The services look separate, but they still depend on each other through the same tables.
• ❌ No timeouts: a cascading failure (one failure causing the next) is waiting to happen (post 41).
• ❌ A cache without a plan for failure: the database dies when the cache does (post 16).
• ❌ A single region, single AZ, single DNS provider, single certificate process: hidden SPOFs (post 40).
• ❌ Replication instead of backups: mistakes copied everywhere (post 45).
• ❌ Secrets in code, trusting the client, unpatched dependencies (posts 51–52).
• ❌ Big-bang deploys without canaries or rollback (post 58).
• ❌ Adding every box on day one: complexity without need (post 6).