Networking for Developers
How the Internet Works: Layers, Packets and Addresses
The layer models, what a packet is, MAC, IP and ports, and following one request from your laptop to a server.
On this page
- Words used in this lesson
- Why we use layers
- Two models: OSI and TCP/IP
- Encapsulation: envelopes inside envelopes
- Three kinds of address
- Ports and sockets: who is listening
- Packet size, MTU and why big messages are split
- Following one HTTP request, step by step
- Latency, bandwidth and the speed of light
- Try it
- Interview questions
- Key takeaways
- What's next
In one line: the internet is a stack of small, simple jobs. One layer moves bits on a wire, one finds a path between machines, one makes a reliable stream, and one speaks HTTP. Each layer wraps the data from the layer above in its own envelope. Once you can see the envelopes, "the network" stops being magic and becomes something you can look at, measure and debug.
Most developers treat the network like electricity. You plug in, it works, and when it does not you restart something. That is fine until the day a request takes 3 seconds only in production, or a service "cannot connect" for no clear reason. On that day, you need a mental picture of what really happens between your fetch() call and the server's res.end().
This series builds that picture, with small experiments you can run on your own computer. This is lesson 0 of 9. You do not need any networking background. You only need to be able to read a little JavaScript and run commands in a terminal. In this lesson we look at the layers, we open a real packet and read its bytes one by one, and then we follow one HTTP request from start to finish. The next lesson, IP Addresses, Subnets, NAT and Routing, zooms in on the "find a path" layer. If you want the browser view of the same journey, Browser Lesson 0 (What happens when you type a URL) is a good companion.
What I ran / checked. Everything here ran on a Linux machine (a sandbox VM) with Node.js v22.22.0, Python 3.13 with scapy 2.8.0, tcpdump 4.99.4 (libpcap 1.10.4), curl 8.5.0 (OpenSSL 3.0.13, nghttp2 1.59.0), ip from iproute2 6.1.0, ss, ping and traceroute. The outputs below are pasted from real runs; the ports, sequence numbers and timings change every time you run them. Sources I read (all in my own words below): the MDN guide "Evolution of HTTP", the MDN guide "Connection management in HTTP/1.x", and the MDN glossary page for TCP, taken as raw markdown from the mdn/content repository on GitHub; and the HTTP/2 specification source (the httpbis revision of RFC 7540, from the httpwg/http2-spec repository). Blocked: rfc-editor.org, ietf.org, datatracker.ietf.org, hpbn.co and cloudflare.com could not be reached from the sandbox, so I did not read the TCP or IP RFCs directly. The header layouts I show come from reading real bytes with a Python script, not from a spec page. Sandbox quirks you should know about: this machine has no IPv6 support, its public DNS always answers "not found", and its network proxy answers TCP connections to random addresses (I saw a handshake with 10.255.255.1 succeed, which a real network would not do). I only used the local network and the local gateway for the examples that matter. I did not verify any GitHub star counts.
Words used in this lesson
- A network is two or more computers that can send data to each other.
- A packet is one small chunk of data with an address on it. Big messages are cut into many packets.
- A protocol is an agreed set of rules for how two sides talk. HTTP, TCP and IP are all protocols.
- A layer is one level of the stack. Each layer does one job and only talks to the layer above and below it.
- Encapsulation means wrapping the data from the upper layer inside the envelope of the lower layer.
- A header is the extra bytes a layer adds in front of your data. They hold the addresses and settings that layer needs.
- A payload is the data inside the envelope, the part the layer above gave you.
- Ethernet is the most common way machines on the same local network send frames to each other (Wi-Fi is similar in spirit).
- A frame is the packet at the lowest layer, with Ethernet addresses on it.
- A MAC address is the hardware address of one network card, six bytes like
02:fc:00:00:00:01. It only matters on the local link. - An IP address is the address of a machine on the wider network, like
192.0.2.2. - A port is a number (0 to 65535) that picks one program on a machine. IP finds the machine, the port finds the program.
- A socket is one end of a connection, an IP address plus a port.
- A gateway (or router) is a machine that passes packets from your network to other networks.
- MTU (maximum transmission unit) is the biggest packet a link will carry.
- Latency is how long one trip takes. Bandwidth is how much data fits through per second.
- RTT (round-trip time) is the time for a message to go out and the reply to come back.
- TTL (time to live) is a counter in every IP packet that goes down by one at each router, so lost packets do not travel forever.
Why we use layers
Imagine you must send a birthday card to a friend in another country. You write the card. You put it in an envelope with the address. The post office puts that envelope into a bigger bag for a plane. The plane company puts the bag in a container. At the other end, each wrapper is taken off in reverse order. You do not need to know how planes work to send a card, and the pilot does not need to read your card.
Networks work the same way. Instead of one giant program that knows everything about cables, routing, retries and web pages, we split the work into layers. Each layer offers a simple promise to the one above:
- The link layer says: "I can deliver a frame to another machine on the same local network."
- The internet layer (IP) says: "I can deliver a packet to any machine in the world, but I do not promise it arrives."
- The transport layer (TCP or UDP) says: "I can deliver data to the right program. TCP also promises order and no missing pieces."
- The application layer (HTTP, DNS, TLS and so on) says: "I know what the bytes mean."
Because of this split, you can change one layer without breaking the others. Your phone can move from Wi-Fi to mobile data (the link layer changes) and the same web page keeps working, because HTTP never knew about Wi-Fi. HTTP/3 replaced TCP with something new underneath HTTP (we will see this in lesson 4) and every website kept its URLs.
Two models: OSI and TCP/IP
You will hear two models. The first one is OSI, with seven layers. It is a teaching model, and people still use its numbers when they talk. The second one is the TCP/IP model, with four layers. It describes what the internet really uses.
| OSI layer | Name | TCP/IP layer | Example |
|---|---|---|---|
| 7 | Application | Application | HTTP, DNS, SSH, SMTP |
| 6 | Presentation | Application | TLS encryption, character encodings |
| 5 | Session | Application | Login sessions, connection setup |
| 4 | Transport | Transport | TCP, UDP, QUIC |
| 3 | Network | Internet | IP, ICMP (ping) |
| 2 | Data link | Link | Ethernet, Wi-Fi, ARP |
| 1 | Physical | Link | Cables, radio, light in fibre |
You do not need to learn the OSI names by heart for daily work. But you must know the numbers that engineers actually say:
- Layer 2 means Ethernet and MAC addresses. "That is a layer 2 problem" means two machines on the same switch cannot see each other.
- Layer 3 means IP. "A layer 3 router" forwards packets by IP address.
- Layer 4 means TCP and UDP, so ports. A "layer 4 load balancer" balances by IP and port without reading HTTP (lesson 6).
- Layer 7 means the application, so HTTP. A "layer 7 load balancer" can look at the URL and headers.
OSI layers 5 and 6 blur in practice. TLS is often drawn at layer 6 but it really sits between TCP and HTTP as a library (lesson 5). That is the main reason people prefer the four-layer picture. Both pictures are only models. Real protocols do not always fit.
Do not fight about layers in an interview. If someone asks "which OSI layer is TLS?", a good answer is "it does not fit neatly, usually called layer 5 or 6, it runs on top of TCP and below HTTP". That answer shows you understand the stack, which is more useful than the number.
Encapsulation: envelopes inside envelopes
When your program sends GET /hi, the bytes go down the stack. Each layer adds its own header in front (and the link layer may add a small trailer at the end).
Your app writes: [ GET /hi HTTP/1.1 ... ] HTTP messageTCP adds a header: [ TCP header | GET /hi HTTP/1.1 ... ] a "segment"IP adds a header: [ IP header | TCP header | GET /hi ... ] a "packet"Ethernet adds one: [ Eth header | IP header | TCP header | GET ... ] a "frame"The names (message, segment, packet, frame) are not used very strictly in daily talk. People say "packet" for all of them. But it helps to know that the thing on the wire is a frame holding a packet holding a segment holding your message.
On the other machine, the reverse happens. Ethernet checks "is this frame for me?" and removes its header. IP checks "is this packet for my address?" and removes its header. TCP checks the port, puts the bytes in order, and hands the stream to the program listening on that port.
Let us stop talking and look at real bytes. I started a tiny HTTP server on port 3000 and captured one request with tcpdump:
// srv.mjsimport http from "node:http";http.createServer((req, res) => { res.setHeader("content-type", "text/plain"); res.end("hello\n"); }) .listen(3000, "127.0.0.1");node srv.mjs &tcpdump -i lo -n -e -vv -X -c 12 'tcp port 3000' > cap.txt &curl -s http://127.0.0.1:3000/hiThe first packet that tcpdump printed was the very first packet of the connection (the "SYN", which asks to open it). It is 74 bytes long. This is the raw hex of everything after the Ethernet header:
0x0000: 4500 003c 063d 4000 4006 367d 7f00 00010x0010: 7f00 0001 c46c 0bb8 d9a5 3678 0000 00000x0020: a002 ffd7 fe30 0000 0204 ffd7 0402 080a0x0030: af6c 0abb 0000 0000 0103 030aThat looks like noise. I wrote a 12-line Python script that cuts it at the boundaries of the two headers and names each field:
import struct, ipaddresshexs = "4500 003c 063d 4000 4006 367d 7f00 0001 7f00 0001 c46c 0bb8 d9a5 3678 0000 0000 a002 ffd7 fe30 0000 0204 ffd7 0402 080a af6c 0abb 0000 0000 0103 030a"b = bytes.fromhex(hexs.replace(" ", ""))ver_ihl, tos, tot, ident, ff, ttl, proto, ck, src, dst = struct.unpack("!BBHHHBBH4s4s", b[:20])print("IP version", ver_ihl >> 4, "header bytes", (ver_ihl & 15) * 4, "total length", tot, "TTL", ttl, "protocol", proto, "(6=TCP)")print(" flags/frag 0x%04x (0x4000 = Don't Fragment)" % ff, "src", ipaddress.ip_address(src), "dst", ipaddress.ip_address(dst))t = b[20:]sp, dp, seq, ack, off_flags, win, ck, urg = struct.unpack("!HHIIHHHH", t[:20])flags = off_flags & 0x1ffnames = [n for bit, n in [(1, "FIN"), (2, "SYN"), (4, "RST"), (8, "PSH"), (16, "ACK")] if flags & bit]print("TCP src port", sp, "dst port", dp, "seq", seq, "ack", ack, "header bytes", (off_flags >> 12) * 4, "flags", names, "window", win)print(" options bytes:", t[20:].hex(" "))IP version 4 header bytes 20 total length 60 TTL 64 protocol 6 (6=TCP) flags/frag 0x4000 (0x4000 = Don't Fragment) src 127.0.0.1 dst 127.0.0.1TCP src port 50284 dst port 3000 seq 3651483256 ack 0 header bytes 40 flags ['SYN'] window 65495 options bytes: 02 04 ff d7 04 02 08 0a af 6c 0a bb 00 00 00 00 01 03 03 0aRead it field by field. This is the real thing, not a drawing:
45is two numbers in one byte. The4means IP version 4. The5means the header is 5 groups of 4 bytes, so 20 bytes.003cis the total length, 60 bytes. That is the IP header (20) plus the TCP header (40). There is no data in a SYN.40in4006is the TTL, 64.06is the protocol number that tells the receiver "the payload is TCP". UDP would be 17.7f00 0001twice: the source and destination addresses,127.0.0.1(each byte is one number,7fis 127).- Then the TCP part starts.
c46cis the source port, 50284.0bb8is the destination port, 3000 (0x0bb8is 3000 in hex). The client got a random high port; the server is on the well-known one we picked. d9a5 3678is the sequence number, a counter for bytes. The next 4 bytes are zero, because there is nothing to acknowledge yet.a002holds the header length (the 10 groups of 4 = 40 bytes) and the flag bits. The only flag set is SYN.ffd7is the window, 65495, which is how much data the sender is ready to receive. Lesson 2 uses this a lot.- The last 20 bytes are TCP options: the maximum segment size, timestamps, a window scale and "I understand selective acknowledgements".
So a SYN is 14 bytes of Ethernet, 20 bytes of IP and 40 bytes of TCP. That is 74, which is the "length 74" that tcpdump printed. The point is not to memorise offsets. The point is that a packet is just bytes with a fixed layout, and any program can read them. Tools like Wireshark and tcpdump do this reading for you.
You can also build a packet. The Python library scapy (pip install scapy) makes the layers look like a stack, with / meaning "put this inside that":
from scapy.all import IP, TCP, Ether, rawp = Ether() / IP(dst="192.0.2.1", ttl=64) / TCP(sport=40000, dport=80, flags="S", seq=1000)print(p.summary()); print(len(raw(p)), "bytes: 14 Ethernet + 20 IP + 20 TCP")Ether / IP / TCP 192.0.2.2:40000 > 192.0.2.1:http S54 bytes: 14 Ethernet + 20 IP + 20 TCPI did not send this one. I only built it to show the layers. (Sending raw packets needs root, and a hand-built SYN confuses the kernel, which did not make that connection.) Notice the packet is 54 bytes and not 74: scapy did not add the TCP options that the real kernel added.
Three kinds of address
A packet carries three kinds of address, and each one answers a different question.
| Address | Example | Question it answers | Lives at | Changes on the way? |
|---|---|---|---|---|
| MAC | 02:fc:00:00:00:01 | Which network card on this local link? | Layer 2 | Yes, at every router |
| IP | 192.0.2.2 | Which machine in the world? | Layer 3 | No (except with NAT, lesson 1) |
| Port | 3000 | Which program on that machine? | Layer 4 | No (except with NAT) |
Let us look at them on a real machine. This is my network card and its addresses:
$ ip -br addrlo UNKNOWN 127.0.0.1/8eth0 UP 192.0.2.2/24docker0 DOWN 172.17.0.1/16$ ip link show eth04: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc pfifo_fast state UP ... link/ether 02:fc:00:00:00:01 brd ff:ff:ff:ff:ff:ffeth0 is the real network card. lo is the loopback interface, a fake one that stays inside the machine. 127.0.0.1 always means "me". The /24 after the address says how big my local network is. Lesson 1 explains that number fully.
Now, how does my machine send something to 192.0.2.1, which is my gateway? The IP packet needs an Ethernet frame, and the frame needs the MAC address of the other side. The machine finds it with ARP ("who has 192.0.2.1? tell 192.0.2.2"), and keeps the answer in a small table:
$ ip neigh192.0.2.1 dev eth0 lladdr 02:fc:00:00:00:05 REACHABLEI captured a single ping to the gateway on eth0 with -e, which tells tcpdump to print the Ethernet header too:
$ tcpdump -i eth0 -n -e -c 2 icmp02:fc:00:00:00:01 > 02:fc:00:00:00:05, ethertype IPv4 (0x0800), length 98: 192.0.2.2 > 192.0.2.1: ICMP echo request, id 13102, seq 1, length 6402:fc:00:00:00:05 > 02:fc:00:00:00:01, ethertype IPv4 (0x0800), length 98: 192.0.2.1 > 192.0.2.2: ICMP echo reply, id 13102, seq 1, length 64Two addresses, two jobs. 02:fc:...:01 > 02:fc:...:05 is who is talking on this wire. 192.0.2.2 > 192.0.2.1 is who is talking end to end. In this case both pairs are about the same two machines, because the gateway is one hop away. Now look at what happens when the destination is far away. traceroute sends packets with a small TTL, so each router in turn answers "your packet died here":
$ traceroute -n -m 4 -w1 1.1.1.1traceroute to 1.1.1.1 (1.1.1.1), 4 hops max, 60 byte packets 1 192.0.2.1 0.504 ms 0.448 ms 0.436 ms 2 21.4.0.169 0.428 ms 0.416 ms 0.406 ms 3 * * * 4 * * *The first router is my gateway. The second hop is some address inside the sandbox's own provider network. Hops 3 and 4 printed * * *, which means no router answered in time (this sandbox filters those replies). Real traceroutes on a normal home connection show 8 to 20 hops. On this path, the IP destination stayed 1.1.1.1 the whole time, but the MAC addresses changed at every hop: my card to the gateway's card, then the gateway's other card to the next router's card, and so on. MAC is "next stop", IP is "final stop".
Do not confuse "the packet's TTL ran out" with a bug. TTL going to zero is exactly how traceroute works. If you see Time to live exceeded from a ping, it only means you set a TTL too small, or that there is a routing loop. I tried ping -c1 -t 1 8.8.8.8 on purpose and got From 192.0.2.1 icmp_seq=1 Time to live exceeded: the gateway threw my packet away, as the rules say.
Ports and sockets: who is listening
IP gets the packet to the machine. The port gets it to the program. A server program asks the operating system "please send me everything that arrives at port 3000". This is called listening. You can see every listener with ss (socket statistics):
$ ss -tlnpState Recv-Q Send-Q Local Address:Port Peer Address:Port ProcessLISTEN 0 511 127.0.0.1:4010 0.0.0.0:* users:(("node",pid=30294,fd=21))LISTEN 0 511 127.0.0.1:4011 0.0.0.0:* users:(("node",pid=30294,fd=22))The flags mean: -t TCP, -l listening only, -n show numbers and not names, -p show the process. This command is the first thing to run when someone says "the port is already in use" (EADDRINUSE in Node).
The address a server listens on matters a lot. 127.0.0.1:4010 means "only connections from this same machine". 0.0.0.0:4010 means "all addresses on this machine, including the network". A very common mistake is to start a server in Docker with localhost, then wonder why nothing outside the container can reach it (the container's localhost is its own).
The client also has a port, picked by the operating system from a high range. Linux shows its range here:
$ cat /proc/sys/net/ipv4/ip_local_port_range32768 60999A connection is named by four numbers: source IP, source port, destination IP, destination port. That is why 1,000 browsers can talk to the same server port 443. Each connection has a different source IP or source port, so the four numbers are all different. This is also why one client machine can only keep about 28,000 connections open to the same server IP and port: it runs out of source ports. (28,232 is the size of the range above: 60999 minus 32768 plus 1.)
Packet size, MTU and why big messages are split
A link cannot carry a packet of any size. Ethernet usually allows 1500 bytes of IP packet, which is the usual MTU. Some links are smaller on purpose, for example when the packet is wrapped in a tunnel (a VPN or a cloud network adds its own header). On this machine, the card says mtu 1400. We can check that with ping. The -M do flag means "do not fragment", -s sets the data size. The packet size is the data plus 28 bytes (20 for IP, 8 for ICMP):
$ ping -c1 -M do -s 1372 192.0.2.1PING 192.0.2.1 (192.0.2.1) 1372(1400) bytes of data.1380 bytes from 192.0.2.1: icmp_seq=1 ttl=64 time=0.386 ms$ ping -c1 -M do -s 1373 192.0.2.1ping: local error: message too long, mtu=1400A 1400 byte packet passes. A 1401 byte packet is refused. This is the MTU, measured, not read from a book.
What happens to a message bigger than the MTU? TCP avoids the problem. During the handshake, each side says "the biggest piece of data I want is X bytes" (this is the MSS, maximum segment size). When I opened a connection from this machine over eth0, the capture showed mss 1360: that is 1400 (MTU) minus 40 (IP header 20 plus TCP header 20). Your 100 KB response is cut into many segments of 1360 bytes or less. On loopback, the MTU is 65536, so the MSS in the first capture was 65495.
If a packet is too big for a link in the middle of the path and the sender said "do not fragment", the router drops it and sends back an ICMP message "fragmentation needed". When those ICMP messages are blocked by a firewall, some connections work for small data and then hang for big data. This failure is famous because it looks random. It shows up with VPNs and some cloud setups. We will use it in lesson 8.
Following one HTTP request, step by step
Now we put the pieces together. You run this in a terminal:
curl http://localhost:4010/hiHere is everything that happens, in order. I used a real local server and curl -w to time the steps.
Name to address (DNS).
localhostis not an address. The program asks the operating system to turn the name into an IP. Forlocalhostthat is found in a local file (/etc/hosts), so it takes microseconds. For a real site, this is a DNS query over the network (lesson 3).Route decision. The OS looks at the destination IP and picks the interface.
ip route getshows the decision:TEXT$ ip route get 8.8.8.88.8.8.8 via 192.0.2.1 dev eth0 src 192.0.2.2$ ip route get 127.0.0.1local 127.0.0.1 dev lo src 127.0.0.1For an outside address, send via the gateway on
eth0. For127.0.0.1, deliver locally. This is lesson 1.Find the next MAC (ARP). To send to the gateway, the OS needs its MAC address. It is usually already in the table (
ip neighabove).TCP handshake. Client and server exchange three packets to agree on starting numbers. This costs one round trip (lesson 2).
TLS handshake (HTTPS only). Keys and certificates are exchanged. Costs one more round trip with TLS 1.3 (lesson 5).
HTTP request and response. Text goes in, text comes out.
Close or reuse. The connection is closed or kept open for the next request.
I measured steps 1, 4, 5 and 6 with curl -w, once for plain HTTP and once for HTTPS (a self-signed certificate I made for the lab):
$ curl -s -o /dev/null -w "dns %{time_namelookup}s tcp-connect %{time_connect}s first-byte %{time_starttransfer}s total %{time_total}s\n" http://localhost:4010/hidns 0.000020s tcp-connect 0.000149s first-byte 0.006410s total 0.006482s$ curl -s -o /dev/null -w "dns %{time_namelookup} connect %{time_connect} tls %{time_appconnect} first-byte %{time_starttransfer} total %{time_total}\n" --cacert cert.pem https://localhost:4011/hidns 0.000032 connect 0.000216 tls 0.005531 first-byte 0.007821 total 0.007991Notice the numbers are cumulative. DNS took 20 microseconds. TCP connect was done at 0.15 ms. In the HTTPS run, TLS was done at 5.5 ms, which is much longer than TCP, because TLS does public-key maths. On loopback every round trip is almost free. On a real long-distance connection, each round trip costs maybe 50 to 150 ms, so a page that needs DNS, TCP, TLS and HTTP in sequence pays four round-trip delays before the first byte. This is the number-one reason performance work in this series is about round trips, not about bandwidth.
Here are all the packets of the plain HTTP request, trimmed to one line each (from the same capture, flags and sequence numbers only):
127.0.0.1.50284 > 127.0.0.1.3000: Flags [S], seq 3651483256, length 0 1 SYN: "can we talk?"127.0.0.1.3000 > 127.0.0.1.50284: Flags [S.], seq 4166820640, ack 3651483257 2 SYN-ACK: "yes"127.0.0.1.50284 > 127.0.0.1.3000: Flags [.], seq 1, ack 1, length 0 3 ACK: "good"127.0.0.1.50284 > 127.0.0.1.3000: Flags [P.], seq 1:80, ack 1, length 79 4 the HTTP request (79 bytes)127.0.0.1.3000 > 127.0.0.1.50284: Flags [.], seq 1, ack 80, length 0 5 "got your 79 bytes"127.0.0.1.3000 > 127.0.0.1.50284: Flags [P.], seq 1:155, ack 80, length 154 6 the HTTP response (154 bytes)127.0.0.1.50284 > 127.0.0.1.3000: Flags [.], seq 80, ack 155, length 0 7 "got your 154 bytes"127.0.0.1.50284 > 127.0.0.1.3000: Flags [F.], seq 80, ack 155, length 0 8 client closes (FIN)127.0.0.1.3000 > 127.0.0.1.50284: Flags [F.], seq 155, ack 81, length 0 9 server closes (FIN)127.0.0.1.50284 > 127.0.0.1.3000: Flags [.], seq 81, ack 156, length 0 10 last ACKTen packets for six bytes of "hello". That is the cost of a clean exchange. The 79-byte request and the 154-byte response were readable text in the hex dump:
GET /hi HTTP/1.1Host: 127.0.0.1:3000User-Agent: curl/8.5.0Accept: */*HTTP/1.1 200 OKcontent-type: text/plainDate: Mon, 12 Oct 2026 03:42:56 GMTConnection: keep-aliveKeep-Alive: timeout=5Content-Length: 6helloThat is the "application layer": plain text sitting inside a TCP segment, inside an IP packet, inside an Ethernet frame. Lesson 2 explains the TCP dance, and lesson 4 explains the HTTP text.
The checksum lines on loopback. If you capture on lo yourself, you will see warnings like cksum 0xfe30 (incorrect -> 0xb594). Do not panic. Your network card normally fills in the checksum at the last moment (this is called checksum offload), and tcpdump sees the packet before that happens. On loopback there is no real card, so the field stays unfilled. It is not a bug in your program.
Latency, bandwidth and the speed of light
Two numbers describe a network connection, and they are easy to mix up.
- Bandwidth is the width of the pipe: megabits per second.
- Latency is the length of the pipe: how long a bit takes to arrive.
A 1 Gbit/s link to the moon is still slow for a chat. Most small web requests (a JSON response of 5 KB) never use the bandwidth. They wait for round trips. The speed of light in fibre is about two thirds of its speed in vacuum, so the minimum RTT between two cities is fixed by distance. Roughly, London to New York takes at least about 55 ms each way, and you cannot buy your way out. The only fixes are to do fewer round trips (keep connections open, use HTTP/2 or HTTP/3, use TLS 1.3) or to put the server closer (a CDN, lesson 6).
I use these rough values as a mental model. They are approximations from general knowledge, not measured here: same data centre well below 1 ms, same continent 10 to 40 ms, across an ocean 70 to 150 ms. If your numbers are much worse, something is wrong, and the rest of this series teaches you where to look.
Try it
- Run
ip -br addr(Linux) orifconfig(macOS), and write down your IP address and the/number next to it. - Run
ip route(ornetstat -rnon macOS). Which address is your default gateway? - Start any local server (for example the 4-line one above) and run
ss -tlnp(orlsof -iTCP -sTCP:LISTEN). What address and port does it listen on? Change127.0.0.1to0.0.0.0and run it again. What changed? - Run
sudo tcpdump -i lo -n -c 10 'tcp port 3000'in one terminal, andcurl http://127.0.0.1:3000/in another. Count the packets and match each one to the ten-line table above. - Run
curl -s -o /dev/null -w "%{time_connect} %{time_total}\n" https://example.comagainst a real site and compare the connect time with the total. Which one is bigger? - Run
traceroute -n example.com(ortracerton Windows). How many hops? Do any lines show* * *?
Answers
- For example
192.168.1.23/24. The/24means the first 24 bits are the network part (lesson 1). - It is the address after
default via. At home it is usually your router, often192.168.0.1or192.168.1.1. - With
127.0.0.1thessoutput shows127.0.0.1:3000, reachable only from the same machine. With0.0.0.0it shows0.0.0.0:3000(or*:3000), reachable from other machines on the network too (if no firewall blocks it). - At least ten packets: three for the handshake, then request, its ACK, response, its ACK, and about three for closing. Some may be missing if the capture ended early (
-c 10stops after ten). - The total is bigger, because it includes TLS and the response. If
time_connectis, say, 40 ms andtime_total150 ms, then TLS and the first response added roughly 110 ms, which is about two or three more round trips. - Usually 8 to 20 hops.
* * *means that router did not send the "time exceeded" reply, which many networks block on purpose. It does not mean the path is broken.
Interview questions
Explain the layers of the network stack and what each one does. (Must know)
I use the four-layer TCP/IP picture. The link layer (Ethernet, Wi-Fi) delivers frames between machines on the same local network, using MAC addresses. The internet layer (IP) delivers packets between any two machines by IP address, with no promise of delivery. The transport layer (TCP or UDP) delivers data to the right program using ports; TCP adds order and reliability. The application layer is where HTTP, DNS and TLS live. Each layer only uses the layer below as a black box, which is why I can change the link (Wi-Fi to mobile) without changing the web app. I also know the OSI numbers people say aloud: layer 2 is Ethernet, 3 is IP, 4 is TCP/UDP, 7 is the application.
What is encapsulation? (Must know)
Each layer wraps the data from the layer above in its own header. HTTP text goes into a TCP segment, which goes into an IP packet, which goes into an Ethernet frame. The receiver unwraps in reverse. I can show it with tcpdump -X: a simple SYN is 14 bytes of Ethernet, 20 of IP and 40 of TCP, so 74 bytes with no data.
What is the difference between a MAC address and an IP address? (Must know)
A MAC address identifies a network card and only matters on one local link. An IP address identifies a machine on the wider network. When a packet crosses routers, the IP addresses stay the same, but the MAC addresses change at each hop, because each router builds a new Ethernet frame to the next stop. MAC is the next stop, IP is the final destination. NAT is the exception that rewrites IP addresses (lesson 1).
Why do we need ports? (Must know)
An IP address only gets the packet to a machine, and one machine runs many programs. The port number picks the program. A connection is identified by four values: source IP, source port, destination IP and destination port. That is why thousands of clients can use the same server port at once, and also why one client IP is limited to around 28,000 connections to one server address and port with the default Linux port range.
A request is slow only in production. How do you think about where the time goes? (Senior signal)
I split it into steps and measure each one: DNS lookup, TCP connect, TLS handshake, time to first byte, and download. curl -w with time_namelookup, time_connect, time_appconnect and time_starttransfer does this in one command. If DNS is slow, I look at the resolver. If connect is slow, the path or distance. If first byte is slow, the server. Because the numbers are cumulative, I subtract one from the next to get each step's cost.
Why is latency often more important than bandwidth for web pages? (Senior signal)
Most requests are small, so they finish long before they fill the link. The time goes into round trips: DNS, TCP handshake, TLS handshake, and the request itself. Light in fibre has a fixed speed, so distance sets a minimum RTT. Faster bandwidth does not help. Fewer round trips (keep-alive, HTTP/2, HTTP/3, TLS 1.3) or a closer server (CDN) does.
What does TTL do in an IP packet? (Must know)
It is a counter that each router lowers by one. When it reaches zero, the router drops the packet and sends an ICMP "time exceeded" message back. It stops lost packets from circling forever when there is a routing loop. traceroute uses it on purpose by sending packets with TTL 1, 2, 3 and so on, so each router in turn reveals itself.
What is the MTU and what goes wrong when it is too small somewhere on the path? (Senior signal)
The MTU is the biggest packet a link carries. Ethernet is usually 1500; on my test machine it was 1400, which I confirmed with ping -M do -s 1372 passing and -s 1373 failing. TCP picks a segment size (MSS) during the handshake to fit. If a router in the middle has a smaller MTU and the ICMP "fragmentation needed" message is blocked by a firewall, small requests work and large ones hang. This "works for small, hangs for big" pattern is the classic sign of an MTU problem, common around VPNs and tunnels.
"localhost works but the other machine cannot connect." Why might that be? (Trap)
The common causes are: the server listens on 127.0.0.1 only (use 0.0.0.0 or a real address); a firewall or security group blocks the port; it runs inside a container and the port is not published; or the client is using the wrong IP. I check with ss -tlnp on the server to see the listening address, then curl from the other machine. The trap is saying "it is a DNS problem" before checking the listening address.
Which layer does a "layer 7 load balancer" work at, and what can it do that layer 4 cannot? (Senior signal)
Layer 7 is the application layer, so it understands HTTP. It can route by URL path or header, terminate TLS, and rewrite headers. A layer 4 balancer only sees IP addresses and ports, so it can only spread connections. Layer 4 is faster and simpler. Layer 7 is smarter and costs more CPU. We cover both in lesson 6.
Is TLS layer 5, 6 or 7? (Trap)
It does not fit the OSI picture well. It runs on top of TCP and below HTTP, so many books call it layer 5 or 6. I explain what it does (encrypts and authenticates the byte stream) and not just a number, because the number differs by textbook.
Key takeaways
- The network is a stack of layers. Each layer wraps data from the layer above in a header (encapsulation).
- Say layer 2 for Ethernet and MAC, 3 for IP, 4 for TCP and UDP, 7 for HTTP and friends.
- MAC address is the next stop on the local link and changes at every hop. IP address is the final stop. Port picks the program.
- A TCP connection is named by four values: source IP and port, destination IP and port.
- A real packet is just bytes. A SYN on my machine was 74 bytes: 14 Ethernet, 20 IP, 40 TCP. You can read it with
tcpdump -Xand a small script. - Packet size is limited by the MTU (1400 on my test machine). TCP picks its segment size (MSS) to fit.
- One HTTP request costs DNS, TCP, optionally TLS, then HTTP. Round trips, not bandwidth, are usually the cost.
- Useful first tools:
ip addr,ip route get,ss -tlnp,ping,traceroute,tcpdump, andcurl -w.
What's next
Next: IP Addresses, Subnets, NAT and Routing, Lesson 1. We will do CIDR maths by hand (what does /26 mean?), check it with Python, see why your laptop has a 192.168.x.x address, and build a toy NAT table to see how one public IP serves a whole house. Practise now: run ip route and write down what each line means.
Published by Jainik · Last updated · How these lessons are made
Keep learning
Related lessons
- What Happens When You Type a URL (The Full Version) · The Browser Contract
- Volumes, Networks and Running Containers Safely · Docker & Kubernetes
- Services, Ingress and Networking in Kubernetes · Docker & Kubernetes
Part of the study guide
Related series