Load Balancers, Proxies, CDNs and Firewalls

When you run node server.js on your laptop and open localhost:3000, there is one hop. In production there might be five: a CDN, a cloud load balancer, a reverse proxy such as nginx, and finally your app, all inside a firewall. Each one can change the request, time it out, cache it, or refuse it. Many bugs live in those gaps: the user's IP address that is always 10.0.3.7, a WebSocket that dies after one minute, a 502 Bad Gateway that your logs never show, an old page that will not go away.

This is lesson 6 of a nine-lesson series. It builds on TLS and certificates (because proxies usually hold your certificate), on HTTP and on TCP. The next lesson, WebSockets and server-sent events, will go through these same proxies and show what they do to long-lived connections. For the security side of servers see Security Lesson 7 (APIs and servers).

Words used in this lesson

• A proxy is a server that sits between two parties and passes messages along.
• A forward proxy works for the client. Clients inside a company send their requests to it to reach the internet.
• A reverse proxy works for the server. Clients think they talk to your site, but they talk to the proxy, which forwards to your app.
• A load balancer spreads requests over several backend servers.
• A backend (or upstream) is a server that sits behind the proxy.
• Layer 4 (L4) means working with TCP connections only: addresses and ports. Layer 7 (L7) means reading HTTP: paths, headers, cookies.
• A health check is a small request sent to a backend on a timer to see if it is fine.
• A sticky session (also called session affinity) means the same client goes to the same backend each time.
• TLS termination means the proxy decrypts HTTPS so the backends see plain HTTP.
• A CDN (content delivery network) is a set of cache servers around the world. Users get content from a server near them.
• An edge server is one of those servers close to the user.
• A cache hit means the answer came from the cache. A miss means the cache had to ask the origin (your server).
• The origin is your real server behind the CDN.
• A firewall allows or blocks network traffic by rules. A security group is the cloud version, attached to a server.
• Gateway errors: 502 Bad Gateway (the proxy got a bad or no answer from the backend), 503 Service Unavailable (no backend can take the request), 504 Gateway Timeout (the backend took too long).

Why put something in front of your app?

A proxy in front of your Node app can do many jobs, so that the app does not have to:

• Terminate TLS: hold the certificate and do the encryption work.
• Spread load across several copies of the app, and avoid sending traffic to sick ones.
• Serve static files and compress responses without waking up your JavaScript.
• Cache answers that do not change.
• Hide the internal network: the outside world sees one address, not your database or your worker machines.
• Limit damage: rate limits, request size limits, timeouts, blocking known bad requests.
• Allow zero-downtime deploys: start new servers, move traffic over, remove the old ones.

The price: another hop (a little latency), another thing that can fail, and another place where behaviour can differ from your laptop. Let us look at each piece, starting with the simplest.

Forward proxy: the client's helper

A forward proxy is configured on the client side. In a company, all laptops may send their web traffic through one proxy that logs, filters, or caches. Your app uses it when you set HTTPS_PROXY or HTTP_PROXY (in my sandbox this was set and it caught me out in the last lesson).

How does a proxy handle HTTPS if it cannot read it? For plain HTTP, the client sends the whole request to the proxy with a full URL. For HTTPS, the client sends one plain request first: CONNECT host:443. That says "please open a TCP tunnel to this host and port". After the proxy replies 200 Connection Established, the client starts the TLS handshake through the tunnel, directly with the real server. The proxy just copies bytes.

I wrote a tiny forward proxy in Node (about 20 lines: an HTTP server for plain requests, and a connect event handler that opens a TCP socket and pipes both ways). Then I made one plain HTTP request and one HTTPS request through it, with a secret in the query string of each:

$ curl -s -x http://127.0.0.1:3128 http://127.0.0.1:3001/secret?token=abc{"backend":"b1","count":1,...}$ curl -s -x http://127.0.0.1:3128 --cacert pki/ca.crt https://app.lab.test:8441/private?token=abchello from good over TLSv1.3-- what the proxy logged:[proxy] plain  GET http://127.0.0.1:3001/secret?token=abc[proxy] CONNECT app.lab.test:8441

Read the two log lines. For plain HTTP, the proxy saw the full URL including the token. For HTTPS, the proxy saw only the host and port. The path, query and body were inside the encrypted tunnel. That is why TLS protects you from your own network operator, and also why some companies install their own CA on laptops and run a proxy that does decrypt (it presents a certificate for each site signed by the company CA). If your code behaves differently inside such a network, the certificate issuer will be the company, not the real one.

Reverse proxy: the server's front door

A reverse proxy is the opposite. The client does not know about it. DNS points your domain at the proxy, the proxy accepts the connection and forwards the request to the app. Here is the smallest useful nginx setup. It listens on port 8080 and passes everything to a Node app on 3001:

events { worker_connections 256; }http {  server {    listen 127.0.0.1:8080;    location / {      proxy_pass http://127.0.0.1:3001;      proxy_set_header Host $host;      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;      proxy_set_header X-Forwarded-Proto $scheme;    }  }}

The three proxy_set_header lines matter. Without them your app sees the proxy, not the user.

What the backend sees, and what it does not

My test backend prints the headers and the network peer for every request. Through the proxy:

{"backend":"b1","count":1,"xff":"127.0.0.1","peer":"127.0.0.1","host":"localhost"}

peer is 127.0.0.1, the proxy's own address, because the TCP connection is from the proxy to the app. The real client address is only in the X-Forwarded-For header (here 127.0.0.1 is the client too, because I ran curl on the same machine). In production, peer is the proxy's private IP, and if you log or rate-limit by req.socket.remoteAddress, every user looks the same. This is the number one beginner bug with proxies: all users blocked at once by a rate limit, because they all share the proxy's address.

Another thing I noticed. In one test config I forgot the Host line, and nginx sent the backend this:

"host":"one"

one is the name of my upstream block. By default nginx sets Host to the upstream name (from my background knowledge of nginx, and I saw it happen). If your app builds URLs from the Host header, or has several sites keyed by host name, it will break. Set proxy_set_header Host $host; unless you have a reason not to.

X-Forwarded-For can be forged

X-Forwarded-For is just a header. A client can send it too. nginx has two common ways to set it:

• $proxy_add_x_forwarded_for: append the connecting address to whatever the client sent.
• $remote_addr: overwrite with the connecting address only.

I sent a request that already claimed X-Forwarded-For: 1.2.3.4 to a location of each kind:

append:     "xff":"1.2.3.4, 127.0.0.1"overwrite:  "xff":"127.0.0.1"

With "append", the app receives a list, and the first entry is whatever the attacker typed. If your code does req.headers['x-forwarded-for'].split(',')[0] to find "the client IP", an attacker can pick their own IP and escape rate limits or fake an audit log. The right way: know how many trusted proxies are in front of you, and take the entry that many places from the right (the proxies add to the end). In Express this is the trust proxy setting (set to the number of proxies, never to plain true unless the edge strips client-supplied values). The edge-most proxy should either overwrite or strip the header. I did not run Express here; check its docs for the exact option for your version.

Also pass X-Forwarded-Proto. When TLS ends at the proxy, your app only sees HTTP. Code that says "if the request is not HTTPS, redirect to HTTPS" will loop forever unless it reads X-Forwarded-Proto from the trusted proxy.

Layer 4 versus layer 7

Load balancers come in two flavours, and the difference is what they read.

• L4 (transport layer). The balancer sees TCP connections only. It picks a backend when the connection opens and then copies bytes both ways. It cannot see the URL, headers or cookies. It is very fast and works for any protocol: databases, MQTT, game servers, TLS passed through without decrypting. Also, one TCP connection stays on one backend for its whole life.
• L7 (application layer). The balancer reads HTTP. It can route /api to one pool and /images to another, add headers, retry a failed request on another backend, cache, rewrite and read cookies. It costs more CPU. And it balances per request, not per connection.

I put the same two backends behind HAProxy twice, once in mode http (port 8090) and once in mode tcp (port 8091), and sent requests to both:

--- L7 (mode http) on 8090: backend sees the forwarded header{"backend":"b2","count":4,"xff":"127.0.0.1","peer":"127.0.0.1","host":"localhost:8090"}--- L4 (mode tcp) on 8091: backend sees only a TCP connection, no XFF header{"backend":"b1","count":8,"xff":null,"peer":"127.0.0.1","host":"localhost:8091"}{"backend":"b2","count":5,"xff":null,"peer":"127.0.0.1","host":"localhost:8091"}

In L7 mode HAProxy added X-Forwarded-For (I turned on option forwardfor). In L4 mode, xff is null: the proxy never looked inside, so it could not add a header. For L4 balancing, the original client address is lost unless you use the PROXY protocol (a small line the balancer writes at the start of the TCP stream) or the balancer preserves the source address at the network level (some cloud balancers do). I did not test the PROXY protocol here.

A common cloud layout is an L4 balancer in front (cheap, fast, handles huge numbers of connections) pointing at a fleet of L7 proxies (smart routing), pointing at your apps.

Balancing algorithms

How does the balancer pick a backend?

• Round robin: take turns. 1, 2, 3, 1, 2, 3. Simple, fair if requests cost about the same.
• Weighted round robin: some backends get more turns. Useful when servers have different sizes.
• Least connections: pick the backend with the fewest active requests. Better when some requests are slow.
• Hash-based (such as nginx ip_hash, or hashing a cookie or URL): the same input always picks the same backend. This gives stickiness and is used for caches.

Real runs against nginx (three backends, ten requests):

round robin, 6 requests:   b1 b2 b3 b1 b2 b3weights 3:1:1, 10 requests (counts):      6 b1      2 b2      2 b3ip_hash, 6 requests from the same client (counts):      6 b3

Round robin did what it says. The 3:1:1 weights split the ten requests as 6, 2, 2. And ip_hash sent all six requests from my one address to the same backend, b3. The problem with ip_hash: everyone behind the same NAT (an office, a mobile carrier) has the same IP address and lands on one server. So it balances badly.

Sticky sessions: usually a bandage

Why would you want a user to return to the same backend? Because the backend keeps something in memory for that user: a login session, a shopping cart, a WebSocket connection. If the user lands on another server, that memory is not there.

Sticky sessions solve this by routing on a cookie or IP, but they have costs: uneven load, and when that server dies or restarts for a deploy, the users' state dies with it. The better design is stateless backends: keep sessions in a shared store (Redis or a database), and let any server handle any request. Use stickiness only when you must, for example WebSocket servers that hold connection state (next lesson).

Health checks: how a balancer knows who is alive

A balancer that keeps sending traffic to a dead server is worse than none. There are two ways to know.

Passive checks watch real traffic. If a request to a backend fails (connection refused, timeout), the balancer marks that backend as failed for a while and tries another. Open-source nginx works this way (max_fails and fail_timeout). I killed backend b2 and sent six requests:

200 b1200 b3200 b3200 b1200 b3200 b1

No errors reached the client. nginx tried b2 on one request, saw the connection refused, and retried the same request on the next backend within the same call. Then it skipped b2 for the next ten seconds. That is the upside of an L7 proxy. The downside is that a real user request paid for finding out.

Active checks are made by the balancer itself on a timer, even with no user traffic. HAProxy does this with option httpchk GET /health and check inter 500ms fall 2 rise 2 (check every half second, two failures to mark it down, two successes to bring it back). My backend's /health returned 503 when I switched it, while the process was still running and still accepting connections:

--- status all upb1 UP   b2 UP   b3 UP--- make b2 unhealthy (the process is alive but /health returns 503)b1 UP   b2 DOWN   b3 UP--- 9 requests: b1 got 4, b3 got 5, b2 got 0--- b2 healthy againb1 UP   b2 UP   b3 UP--- kill the b1 process completelyb1 DOWN   b2 UP   b3 UP

(The uneven 4 and 5 is because the round-robin counter was in the middle of its turn.) Two things to see. First, the health check noticed a backend that was running but sick, which a plain TCP connect check cannot. Second, the check protected users: b2 received no traffic while it was down, with no failed requests.

Retries can hurt

Retries hide failures, but they multiply load and can repeat actions. I set a 1-second read timeout in nginx and called an endpoint that takes 1.5 seconds, with several backends:

slow endpoint, 3 backends (one already dead): 502 after 2.0 s    (nginx waited 1 s, retried on the next backend, waited 1 s again, then gave up)slow endpoint, 1 backend:   504 after 1.0 s

On a timeout nginx, by default, tries the next server (the proxy_next_upstream setting; I saw it retry here, which matches how I understand the default to work, but check the docs). Think about what happens if the endpoint was "charge the customer" and the first backend actually did the work slowly: the retry charges again. By default nginx does not retry non-idempotent requests like POST after they were sent (from my background knowledge, not tested here), and you should keep it that way. The real fix is to make your own endpoints idempotent, with an idempotency key, so that a retry is safe.

Reading gateway errors

The status codes tell you where to look:

The proxy's error log says more than the status. For example (real lines):

connect() failed (111: Connection refused) while connecting to upstream, ... upstream: "http://127.0.0.1:3002/slow"upstream timed out (110: Connection timed out) while reading response header from upstream, ...no live upstreams while connecting to upstream, ... upstream: "http://rr/x"

Notice that your app's own logs show nothing for a 502 or a timeout of the "connection refused" kind, because the request never arrived. Always read the proxy's logs first. Also, set timeouts on purpose: the proxy's read timeout should be a little above your slowest legitimate response, and your app should have its own timeout for each outgoing call that is lower than the proxy's.

Caching and CDNs

A cache stores a copy of a response so the next request does not need the origin. A CDN is a big shared cache with many locations. When a user in Mumbai asks for logo.png, a nearby edge server answers; only the first request, or an expired one, travels to your origin. Benefits: lower latency (shorter distance), less origin load, protection from traffic spikes, and edge TLS (the handshake from last lesson is made with a nearby server, so its round trips are short).

I could not run a real CDN in this sandbox, but nginx's proxy_cache follows the same rules, so here is the same behaviour with real output. My backend sends Cache-Control: public, max-age=5. I made three requests, waited six seconds and made a fourth, printing nginx's cache status header each time:

X-Cache-Status: MISSX-Cache-Status: HITX-Cache-Status: HIT(after 6 seconds)X-Cache-Status: EXPIRED

The first request was a miss and went to the backend. The next two were hits: the backend counter did not move. After five seconds the copy was stale, so the cache asked the origin again (expired) and refreshed its copy. That is the whole idea of web caching in four lines.

What you control, as a developer, is the response headers:

• Cache-Control: public, max-age=3600 means "any cache may keep this for an hour". private means "only the user's own browser", never a shared cache. no-store means "do not keep it". no-cache means "you may keep it but check with me before using it" (a name people get wrong).
• Vary: Accept-Encoding means "the response depends on this request header, so the cache key must include it". Without correct Vary, a cache can serve a compressed page to a client that cannot read it, or one user's language to another.
• ETag / Last-Modified let a cache ask "has it changed?" and get a tiny 304 Not Modified back.
• Cache key: by default, usually method plus URL (plus the Vary headers). Putting a user id in a query string makes every user a separate cache entry and the hit ratio falls to zero.

When you deploy new files, how do you get rid of old cached ones? Two ways: purge (an API call to tell the CDN to forget a URL) or versioned names (app.3f9a2c.js, so a new file has a new URL and the old one can be cached for a year). Versioned names are simpler and more reliable, and are what most frontend build tools produce. For the dynamic API, many teams still use short max-ages and stale-while-revalidate ideas, but I did not test those here.

CDNs also provide DDoS absorption, a web application firewall (WAF), bot filters, image resizing and edge functions. I list them from general knowledge. I did not read a CDN vendor's docs in this session, so check your provider's documentation for what is available and for the exact header names (they differ by vendor).

Firewalls and security groups

A firewall decides which packets may pass, by rules on source and destination address, port and protocol, and direction. Cloud security groups are firewalls attached to a server or network interface. They are normally stateful: if a connection out was allowed, the answer back is allowed automatically. And they are default deny: nothing is allowed unless a rule says so.

Good rules for a typical web app:

• Only the load balancer is open to the internet, on ports 80 and 443.
• App servers accept traffic only from the load balancer (a rule that names the balancer's security group, not "anywhere").
• The database accepts traffic only from the app servers on its own port, and never from the internet.
• SSH is limited to a known address or replaced by a managed access tool.
• Restrict outbound traffic too where you can. A compromised server that cannot call out is harder to use.

What a firewall looks like from the client

This part matters for debugging, so I tested it with iptables on my machine. A firewall can refuse in two ways:

• REJECT: send back an immediate "no" (a TCP reset). The client fails fast.
• DROP: say nothing. The client waits for its own timeout.

open port:                       200 in 0.011 siptables ... -j REJECT --reject-with tcp-reset   curl: exit 7  (connection refused)  in 0.0002 siptables ... -j DROP   curl: exit 28 (timed out)           after 4.0 s  (I set -m 4)nothing listening at all:        curl exit 7 (connection refused), instantly

So connection refused and a long silent timed out mean different things. A refusal says "the machine is up, but nothing is listening on that port (or a firewall actively rejected you)". A timeout says "my packets disappeared": most often a firewall or security group that drops, a wrong IP, or a network route problem. Most cloud security groups use the DROP style, so a forgotten rule shows up as a hang, not as an error. I captured both cases with tcpdump (full details in lesson 8): refused shows a single reset from the server, and dropped shows the client's SYN being sent again and again about one second apart with no answer at all.

A firewall at the network edge is not a replacement for authentication in the app. A WAF (web application firewall) works at layer 7 and tries to block known attack patterns, but it can be bypassed and gives false positives. Treat it as one layer, like in Security Lesson 0.

Putting it together: one request through a real stack

Browser  |  DNS: shop.example.com -> CDN address  vCDN edge (TLS ends here for the browser; cache hit? -> reply)  |  miss -> new connection to origin  vCloud L4/L7 load balancer (health checks, TLS to origin, X-Forwarded-For added)  vReverse proxy (nginx: gzip, static files, timeouts, rate limit)  vNode app (reads trusted X-Forwarded-* headers)  vDatabase (private subnet, only the app can reach it)

At every arrow ask four questions: what does this hop terminate? (TCP, TLS) What does it add or remove? (headers) How long does it wait? (timeouts) What does it cache or retry? Most production network bugs answer one of those four.

Try it

You need nginx or HAProxy (apt install nginx-light haproxy) and Node 22.

1. Write a Node server that answers { name, count } with its name from a command line argument. Start two copies on ports 3001 and 3002.
2. Configure nginx with an upstream of both and a location / that proxies to it. Send 6 requests with curl. What pattern do you see?
3. Kill one backend and send 6 more. Do the clients see errors? What does the nginx error log say?
4. Add proxy_set_header X-Forwarded-For $remote_addr; and print req.headers['x-forwarded-for'] in the backend. Then send curl -H 'X-Forwarded-For: 9.9.9.9' .... What does the backend see? Switch to $proxy_add_x_forwarded_for and try again.
5. Add proxy_cache with a 5-second proxy_cache_valid and an add_header X-Cache-Status $upstream_cache_status;. Request the same URL four times. Which are hits?
6. With sudo iptables -A INPUT -i lo -p tcp --dport 3001 -j DROP (and later iptables -F INPUT), call port 3001 with curl -m 3. Then use -j REJECT --reject-with tcp-reset. Compare the time to fail.

Answers

2. Alternating: backend 1, 2, 1, 2, 1, 2 (round robin is the default).
3. No errors for the clients. nginx retries the request on the live backend. The error log has connect() failed (111: Connection refused) while connecting to upstream.
4. With $remote_addr the backend sees only the real connecting address. With $proxy_add_x_forwarded_for it sees 9.9.9.9, <real address>, so the first entry is the one the client invented.
5. The first request is a MISS; the next ones within five seconds are HITs; after five seconds you see EXPIRED (a revalidation).
6. DROP hangs until curl's own timeout (3 s) and exits with code 28. REJECT fails within milliseconds with exit code 7.

Interview questions

Key takeaways

• A forward proxy works for clients, a reverse proxy works for servers. HTTPS goes through a forward proxy as a CONNECT tunnel, so the proxy sees only host and port.
• Behind a proxy, the TCP peer is the proxy. The real client IP is in X-Forwarded-For, which clients can forge: take the entry that your trusted proxies added, from the right.
• Set Host and X-Forwarded-Proto on purpose, or the app will see the wrong host name or loop on HTTPS redirects.
• L4 balancers move bytes and are fast. L7 balancers read HTTP and can route, retry and cache.
• Round robin, weights, least connections and hashing are the main algorithms. IP hash is poor behind NAT. Prefer stateless backends over sticky sessions.
• Passive health checks learn from failures. Active checks probe on a timer. Keep the health endpoint shallow.
• 502 is a bad or missing reply, 503 is no capacity, 504 is a timeout. Read the proxy's error log first.
• Retries can repeat actions. Make endpoints idempotent.
• A CDN is a shared cache driven by your response headers. Never let personalised responses be cached by a shared cache.
• Firewalls that REJECT fail fast; firewalls that DROP cause hangs. A hang usually means "blocked", a refusal means "nothing is listening".

What's next

Next: WebSockets, Server-Sent Events and Real-Time Networking. A normal HTTP request ends when the response ends. Real-time features need a connection that stays open. You will compare polling, long polling, WebSockets and SSE with real code and measurements, see the exact upgrade handshake, watch a client reconnect and resume after a dropped connection, and see why the proxy settings from this lesson decide whether a WebSocket survives. Practise now: run the nginx cache experiment in "Try it", because the same proxy is the next lesson's main character.