Networking for Developers

DNS: How Names Become Addresses

Resolvers, root and authoritative servers, record types, TTL and caching, and debugging DNS with dig.

Lesson 0314 min read#networking#dns

On this page

  1. Words used in this lesson
  2. One name, three kinds of server
  3. Walking the tree by hand with dig
  4. What a recursive resolver does for you
  5. Record types you will actually meet
  6. Answers that are not addresses: NXDOMAIN, NODATA, SERVFAIL
  7. TTL and caching: the thing that bites
  8. DNS on the wire
  9. How your program asks: it is not always DNS
  10. Debugging DNS with dig
  11. A short word on security
  12. Try it
  13. Interview questions
  14. Key takeaways
  15. What's next

In one line: DNS is a worldwide, cached phone book that turns a name like www.example.com into an IP address. A resolver asks the root servers, then the top-level domain servers, then the domain's own authoritative server, and every answer carries a TTL that says how long it may be remembered. Most "DNS problems" are really caching problems, and dig shows you which server said what.

Computers talk to IP addresses. People remember names. DNS (the Domain Name System) is the bridge, and it runs before almost every connection you make, so it is a quiet source of slowness and outages. Typical stories: "I changed the record an hour ago and half the users still go to the old server", "it works on my laptop but not in the container", "the site is up but one office cannot reach it", "ENOTFOUND in Node, but ping works". After this lesson you will know where each of those comes from.

This is lesson 3 of a nine-lesson series. In lesson 2 we saw how TCP and UDP carry data between IP addresses. DNS is what finds the address in the first place, and it mostly rides on UDP. The next lesson, HTTP/1.1, HTTP/2 and HTTP/3, starts after the name is already resolved.

What I ran / checked. Everything ran on a Linux sandbox (kernel 6.18.44) as root, with dig 9.18.39 (DiG 9.18.39-0ubuntu0.24.04.7-Ubuntu), Python 3.13.16 with the dnslib 0.9.26 library, Node.js v22.22.0 and tcpdump 4.99.4. The real internet was not usable for DNS: the sandbox's configured resolver (8.8.8.8) answered my example.com query with NXDOMAIN, and so did dig +trace, which never reached real root servers, and no DNS server software such as dnsmasq, unbound or BIND is installed. So I built a small DNS system myself in about 150 lines of Python: a fake root server (127.0.0.2), a fake .test top-level server (127.0.0.3), an authoritative server for example.test (127.0.0.4) and a caching resolver (127.0.0.1). All of them listen on port 53 on loopback addresses. They are real DNS servers: dig and Node talk to them with normal DNS packets. They are also toys: no DNSSEC, the resolver is single threaded, and it does not cache the NS delegations (a real resolver does). Everything pasted below comes from real runs; timings vary. For two experiments I temporarily appended a few lines to /etc/hosts and then put the original file back. Sources I read: the MDN glossary pages for DNS and for TTL (short definitions, I used them only to check my wording). Could not reach: the RFCs (1034, 1035, 2181) on rfc-editor.org and datatracker.ietf.org were blocked by the sandbox proxy, so protocol facts that I do not show in a capture come from my background knowledge, and I mark them. Could not run: dig +trace against the real root servers, anything about DNSSEC, DNS over HTTPS and DNS over TLS, and real CDN or cloud DNS behaviour.

Words used in this lesson

  • DNS is the system that maps names to data, mostly IP addresses.
  • A domain name is a dot-separated name like www.example.test. You read it from right to left: test is the top, example is inside it, www is inside that.
  • A label is one piece between dots. An FQDN (fully qualified domain name) is the full name. A trailing dot, as in example.test., means "the complete name, ending at the root".
  • A zone is the part of the name tree that one organisation manages. A zone file holds its records.
  • A record (RR, resource record) is one fact: a name, a type, a TTL and a value.
  • A nameserver is a server that answers DNS questions. An authoritative server holds the real records for a zone and gives the final answer. A recursive resolver does the hunting for you and keeps a cache.
  • The stub resolver is the small client code on your own machine that sends your question to a recursive resolver.
  • The root servers know where the top-level domains (com, org, test, ...) are.
  • A TLD is a top-level domain, like com.
  • A referral is an answer that says "I do not know, but ask these servers". Glue is the IP address that comes with a referral so you can reach those servers.
  • TTL (time to live) is the number of seconds an answer may be cached.
  • A cache is a stored copy of an earlier answer.
  • A record type says what kind of fact a record is: A, AAAA, CNAME, MX, TXT, NS, SOA.
  • NXDOMAIN means "this name does not exist". NODATA means "the name exists, but not with that record type". SERVFAIL means "the server could not get an answer".
  • dig is the standard command line tool for asking DNS questions and reading the full answer.

One name, three kinds of server

Take www.example.test. Read it from the right. There is a hidden top, the root, written as a final dot. Then test, then example, then www. DNS is a tree with the same shape, and different organisations run different parts of it:

LevelWho knows whatIn my lab
RootWhich servers handle each top-level domain127.0.0.2
TLD (test)Which servers handle each domain inside it127.0.0.3
Authoritative (example.test)The real records for the domain127.0.0.4
Recursive resolverNothing of its own. It asks the three above and caches the answers127.0.0.1

The first three only know their own level and point downward. A resolver asks the root, "who handles test?", asks that server, "who handles example.test?", and asks the last one for the real answer. The pointing-down answers are called referrals. On the real internet there are 13 named root server addresses, each served by many machines. I know that from background reading, and I could not check it here.

The zone data for the lab is plain text. This is the interesting part of the authoritative zone (the root and TLD zones hold only the NS and glue lines):

TEXT
example.test.        3600 IN SOA   ns1.example.test. admin.example.test. 7 3600 600 86400 120example.test.        3600 IN NS    ns1.example.test.ns1.example.test.    3600 IN A     127.0.0.4example.test.          60 IN A     192.0.2.10example.test.         300 IN MX    10 mail.example.test.example.test.         300 IN TXT   "v=spf1 ip4:192.0.2.0/24 -all"mail.example.test.    300 IN A     192.0.2.25www.example.test.     300 IN CNAME web.example.test.web.example.test.      60 IN A     192.0.2.10web.example.test.      60 IN A     192.0.2.11web.example.test.      60 IN AAAA  2001:db8::10app.example.test.      20 IN A     (read from a file, so I can change it while the lab runs)

Each line is one resource record: name, TTL in seconds, class (IN for internet, always), type, value. The addresses 192.0.2.x and 2001:db8:: are ranges reserved for examples, so they never clash with real servers.

Walking the tree by hand with dig

dig normally asks a recursive resolver, which hides the steps. With +norec (no recursion desired) I can ask one server one question and see exactly what it knows. Step 1, the root:

TEXT
$ dig @127.0.0.2 www.example.test +norec;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1;; QUESTION SECTION:;www.example.test.		IN	A;; AUTHORITY SECTION:test.			172800	IN	NS	a.nic.test.;; ADDITIONAL SECTION:a.nic.test.		172800	IN	A	127.0.0.3

The root does not know the address. It says: "for names under test, ask a.nic.test, whose address is 127.0.0.3". That is a referral with glue. Notice the flags: qr (this is a response) but no aa, so the server is not claiming to be the authority for this name. Step 2, the TLD server:

TEXT
$ dig @127.0.0.3 www.example.test +norec;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1;; AUTHORITY SECTION:example.test.		172800	IN	NS	ns1.example.test.;; ADDITIONAL SECTION:ns1.example.test.	172800	IN	A	127.0.0.4

Another referral: "ask ns1.example.test at 127.0.0.4". Step 3, the authoritative server:

TEXT
$ dig @127.0.0.4 www.example.test +norec;; flags: qr aa; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0;; ANSWER SECTION:www.example.test.	300	IN	CNAME	web.example.test.web.example.test.	60	IN	A	192.0.2.10web.example.test.	60	IN	A	192.0.2.11

Now there is an ANSWER SECTION and the aa flag (authoritative answer). The answer is a CNAME (an alias, "www is another name for web") followed by the two addresses of web. Read a dig reply with these four parts in mind:

  • flags: qr = response, aa = from the authority, rd = recursion desired (the client asked for it), ra = recursion available (the server offers it), tc = truncated.
  • ANSWER: the records that answer the question.
  • AUTHORITY: records that say who is responsible (in a referral, the NS records; in an error, the SOA).
  • ADDITIONAL: helpful extras such as glue.

dig can follow these steps for you with +trace. It starts with a root server and follows referrals itself, printing each hop. I pointed it at my lab root with @127.0.0.2:

TEXT
$ dig @127.0.0.2 +trace www.example.test +nodnssec.			86400	IN	NS	a.root.;; Received 36 bytes from 127.0.0.2#53(127.0.0.2) in 0 mstest.			172800	IN	NS	a.nic.test.;; Received 70 bytes from 127.0.0.2#53(a.root) in 4 msexample.test.		172800	IN	NS	ns1.example.test.;; Received 68 bytes from 127.0.0.3#53(a.nic.test) in 0 mswww.example.test.	300	IN	CNAME	web.example.test.web.example.test.	60	IN	A	192.0.2.10web.example.test.	60	IN	A	192.0.2.11;; Received 84 bytes from 127.0.0.4#53(ns1.example.test) in 0 ms

Four hops: root, then TLD, then authoritative. (For this to work, dig must be able to turn the names a.root, a.nic.test and ns1.example.test into addresses, so I added three temporary lines to /etc/hosts. On the real internet the lookups of the NS names are done in the normal way.) Against the real internet you would write dig +trace www.example.com. I could not run that here.

What a recursive resolver does for you

You almost never talk to the root yourself. Your computer sends one question to a recursive resolver (your router, your company, your provider, or a public one such as 8.8.8.8). It does the three hops and remembers the result. Here is what my lab resolver logged while dig @127.0.0.1 www.example.test ran, then a second dig, then a third one three seconds later (the left column is seconds since the lab started):

TEXT
[  1.978s] resolver  client asks www.example.test. A[  1.978s] resolver  -> asks 127.0.0.2: www.example.test. A[  1.979s] root      www.example.test. A[  1.979s] resolver  -> asks 127.0.0.3: www.example.test. A[  1.980s] tld       www.example.test. A[  1.981s] resolver  -> asks 127.0.0.4: www.example.test. A[  1.982s] auth      www.example.test. A[  1.993s] resolver  client asks www.example.test. A[  1.993s] resolver  CACHE HIT www.example.test. A (60s left)[  5.017s] resolver  client asks www.example.test. A[  5.017s] resolver  CACHE HIT www.example.test. A (57s left)

The first question needed three upstream queries. The next two were answered from memory with no upstream traffic. The client side shows the same thing:

TEXT
First dig:   ;; Query time: 4 msecSecond dig:  ;; Query time: 0 msecwww.example.test.   59  IN  CNAME  web.example.test.     <- first answerweb.example.test.   59  IN  A      192.0.2.10web.example.test.   59  IN  A      192.0.2.11(3 seconds later, the TTL column counts down)www.example.test.   56  IN  CNAME  web.example.test.

The TTL column is not fixed. It shows how many seconds are left for that cached entry. My toy resolver used the smallest TTL in the answer (60 from the A records, even though the CNAME says 300) and counted down. Real resolvers keep a TTL per record. A real resolver also caches the NS referrals, so a second lookup in the same zone skips the root and the TLD. Mine does not, which is why it asks the root again for each new name.

Record types you will actually meet

I asked the lab for each type:

TEXT
$ dig @127.0.0.1 example.test MX +noall +answerexample.test.		299	IN	MX	10 mail.example.test.$ dig @127.0.0.1 example.test TXT +noall +answerexample.test.		299	IN	TXT	"v=spf1 ip4:192.0.2.0/24 -all"$ dig @127.0.0.1 example.test NS +noall +answerexample.test.		3599	IN	NS	ns1.example.test.$ dig @127.0.0.1 example.test SOA +noall +answerexample.test.		3599	IN	SOA	ns1.example.test. admin.example.test. 7 3600 600 86400 120
TypeWhat it holdsNotes
AAn IPv4 addressThe most common one. A name can have several A records, and the client may use any of them (a poor man's load balancing).
AAAAAn IPv6 addressPronounced "quad A".
CNAME"This name is an alias for that name"The resolver restarts the lookup at the target. The name with a CNAME cannot hold other records. Because the zone's top name (example.com) must carry SOA and NS records, you cannot put a CNAME there. That is the rule I know from background reading, which I did not test. Some DNS providers add a special "ALIAS" or "flattening" feature to work around it.
MXMail servers for the domain, with a priority numberLower number means "try first". Here 10 mail.example.test.
TXTFree textUsed for SPF (which servers may send mail for you), domain verification, and the ACME DNS-01 challenge from lesson 5.
NSThe nameservers for a zoneThis is what referrals are made of.
SOAStart of authority: the zone's main server, an admin contact, a serial number and timersThe last number (120 here) is used as the negative caching time, see below.

Two more you will see in practice, which I did not test in the lab: PTR (reverse lookup, address to name, used by mail servers and logs) and SRV (host and port for a service, used by Kubernetes and some protocols).

Multiple answers. My zone gives web two A records. The client gets both, and decides which one to use. Many programs just take the first, and some resolvers rotate the order between answers. My toy does not rotate, so I cannot show that. The practical lesson: DNS gives you a list, and your code (or HTTP client) should try the next address if the first one fails.

Answers that are not addresses: NXDOMAIN, NODATA, SERVFAIL

The status in the header says how the lookup went. I produced three of them from the lab.

NXDOMAIN, the name does not exist:

TEXT
$ dig @127.0.0.4 nothere.example.test;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0;; AUTHORITY SECTION:example.test.		3600	IN	SOA	ns1.example.test. admin.example.test. 7 3600 600 86400 120

(The status line is status: NXDOMAIN; I cut it from this block to keep it short.) The authority attaches the SOA record to prove it is speaking for the zone. The resolver can use that SOA to remember the "no" as well. Asking the lab resolver for nothere.example.test twice:

TEXT
first:  status: NXDOMAIN   ;; Query time: 4 msecsecond: status: NXDOMAIN   ;; Query time: 0 msec

The second answer came from the cache. This is negative caching: the "does not exist" answer is stored for the SOA's last number (120 seconds in my zone). It helps with a very common mistake: if you query a name before you create it, resolvers remember the "no" and keep returning it for a while after you add the record. The fix is to wait, or to avoid querying the name early.

NODATA, the name exists but has no record of this type. mail.example.test has an A record but no AAAA:

TEXT
$ dig @127.0.0.1 mail.example.test AAAA;; ->>HEADER<<- opcode: QUERY, status: NOERROR;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

The status is NOERROR but the answer section is empty. That is NODATA. It is easy to mistake for a failure, but it is a clean "no such record".

SERVFAIL, the resolver could not find out. I added a fake delegation: broken.test points to a server at 127.0.0.9, where nothing runs.

TEXT
$ dig @127.0.0.1 www.broken.test +tries=1 +time=5;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0;; Query time: 1004 msec[  1.979s] resolver  -> asks 127.0.0.9: www.broken.test. A[  2.980s] resolver  upstream failed: TimeoutError

The resolver asked the dead server, waited a full second (my chosen timeout), and gave up with SERVFAIL. Notice that a UDP question to a port where nobody listens produced a timeout, not a quick error. That is exactly the "unconnected UDP" behaviour from lesson 2. The practical meaning of the three:

StatusMeaningWhere to look
NXDOMAINThe name does not exist in the zone (or you mistyped it)Typo? Record created? Negative cache?
NOERROR with empty answerThe name exists, but not this typeAsked for AAAA but only A exists?
SERVFAILThe resolver failed to get a good answerThe authoritative servers are down or broken, or (with DNSSEC) a signature check failed
REFUSEDThe server will not answer youWrong server asked, or a policy blocks you

TTL and caching: the thing that bites

Every record has a TTL, and every cache between you and the authoritative server may keep the answer for that many seconds. I tested what that does with a record whose TTL is 20 seconds, and I changed its value while the resolver had it cached. The resolver is 127.0.0.1, the authority is 127.0.0.4:

TEXT
t0: first lookupapp.example.test.   19   IN   A   192.0.2.100-- operator changes the record to 192.0.2.200t+5:    app.example.test.   14   IN   A   192.0.2.100     <- still the old valuet+15:   app.example.test.    4   IN   A   192.0.2.100     <- still the old valuet+22:   app.example.test.   19   IN   A   192.0.2.200     <- TTL ran out, new value--- asking the authoritative server directly, any time after the change:app.example.test.   20   IN   A   192.0.2.200

The authority knew the new value instantly. The resolver kept serving the old value until the 20 seconds ran out. Now picture the same thing with a TTL of 86400 (one day) and thousands of resolvers around the world, each with its own countdown. Nobody can tell them to forget. That is why DNS changes are not instant, and why "propagation" is the wrong word: nothing is pushed. Old copies simply expire.

Practical consequences:

  • Before a planned move, lower the TTL first. A day or two before you change an IP, set the TTL to 60 or 300 seconds. Wait until the old TTL has passed. Then change the address. After it is stable, raise the TTL again. This is the standard migration trick.
  • A low TTL costs something. Every expiry is another lookup, more load on your nameservers, and more time before the first byte for users. A TTL of 60 seconds is fine for a load balancer name that changes, a day is fine for something that never moves.
  • There are many caches, not one. In order: your program (some runtimes cache), the operating system's stub resolver (on some systems), your router, your provider's or company's resolver, and the browser (it keeps its own DNS cache, which I know from background knowledge, and did not test). When a user "still sees the old IP", a different layer may hold it, and each has a separate countdown. Ask the authoritative server directly (dig @ns1... name) to see the truth, and ask the user's resolver to see what they see.
  • Negative answers are cached too (see above).
  • Not every cache obeys TTL perfectly. Some old software and some resolvers hold records longer than asked, or clamp very low TTLs up to a minimum. I know this from experience and I did not measure it here.

Do not cache DNS forever inside your own app. A long-lived process that resolves a name once at start-up and keeps the IP will keep talking to the old server after a failover. This hits JVM-based programs (they have their own DNS cache setting) and any code that stores the resolved IP. Resolve per connection or respect the TTL, and let connection pools recycle connections.

DNS on the wire

DNS is a small, binary protocol. Normally it travels in a single UDP packet to port 53, one question and one answer. I built a query by hand in 12 lines of Python to show that there is no magic:

Python
import socket, structdef build(name, qtype=1, qid=0x1234):    hdr = struct.pack(">HHHHHH", qid, 0x0100, 1, 0, 0, 0)   # id, flags (RD), 1 question, 0, 0, 0    qn = b"".join(bytes([len(p)]) + p.encode() for p in name.split(".")) + b"\x00"    return hdr + qn + struct.pack(">HH", qtype, 1)           # type A, class INq = build("web.example.test")s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM); s.settimeout(2)s.sendto(q, ("127.0.0.4", 53)); r = s.recv(512)
TEXT
query bytes: 3412 34 01 00 00 01 00 00 00 00 00 00 03 77 65 62 07 65 78 61 6d 70 6c 65 04 74 65 73 74 00 00 01 00 01reply: id=0x1234 flags=0x8500 (QR=1 AA=1 RCODE=0) questions=1 answers=212 34 85 00 00 01 00 02 00 00 00 00 03 77 65 62 07 65 78 61 6d 70 6c 65 04 74 65 73 74 00 00 01 00 01c0 0c 00 01 00 01 00 00 00 3c 00 04 c0 00 02 0a   c0 0c 00 01 00 01 00 00 00 3c 00 04 c0 00 02 0b

The whole question is 34 bytes. Decode it:

  • 12 34 is the query ID. The reply repeats it, so the client can match answers to questions.
  • 01 00 are the flags (RD, recursion desired). 00 01 is "one question", followed by zeros for answers, authority and extras.
  • The name is stored as length-prefixed labels: 03 w e b, 07 e x a m p l e, 04 t e s t, then 00 for the root. There are no dots on the wire.
  • 00 01 00 01 is type A, class IN.

In the reply, the flags 85 00 mean QR (response) and AA (authoritative). an=2 means two answers. Each answer starts with c0 0c, which is a compression pointer: "the name is at byte 12", to avoid writing web.example.test again. Then type 1, class 1, TTL 00 00 00 3c (60), data length 00 04, and the 4 bytes of the address: c0 00 02 0a is 192.0.2.10 and ...0b is 192.0.2.11. Compact, and easy to parse by hand.

Big answers: truncation and TCP

A plain DNS reply over UDP was historically limited to 512 bytes (that limit I know from background reading; newer clients negotiate more with an extension called EDNS). What if an answer is bigger? I made a TXT record of 1000 bytes and made my authoritative server follow the old rule. First, plain UDP only, and I told dig not to retry:

TEXT
$ dig @127.0.0.4 big.example.test TXT +bufsize=512 +noedns +notcp +ignore;; flags: qr aa tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0;; MSG SIZE  rcvd: 34

The tc flag means truncated: "the answer did not fit, ask again over TCP". Without +notcp, dig does exactly that on its own:

TEXT
$ dig @127.0.0.4 big.example.test TXT +bufsize=512 +noedns;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0;; SERVER: 127.0.0.4#53(127.0.0.4) (TCP);; MSG SIZE  rcvd: 1050

And the capture shows the UDP question and the truncated reply, and then a full TCP connection (the same SYN, SYN-ACK, ACK from lesson 2) that fetches the 1050 byte answer:

TEXT
IP 127.0.0.1.35958 > 127.0.0.4.53: 18315+ [1au] TXT? big.example.test. (57)IP 127.0.0.4.53 > 127.0.0.1.35958: 18315*-| 0/0/0 (34)IP 127.0.0.1.45535 > 127.0.0.4.53: Flags [S], seq 286477565, ...IP 127.0.0.4.53 > 127.0.0.1.45535: Flags [S.], seq 3532498623, ack 286477566, ...IP 127.0.0.1.45535 > 127.0.0.4.53: Flags [P.], seq 1:60, ack 1, ..., length 59IP 127.0.0.4.53 > 127.0.0.1.45535: Flags [P.], seq 1:1053, ack 60, ..., length 1052

(The * in tcpdump means authoritative answer, and | means truncated. Some lines are shortened.) So DNS does use TCP, for big answers, and for zone transfers between servers. This matters in a firewall: allow both UDP and TCP on port 53, or large answers will fail in a confusing way.

How your program asks: it is not always DNS

When your code calls fetch("https://example.com") or socket.connect("db.internal", 5432), the name goes through the operating system's lookup function (getaddrinfo on Linux). It does not always ask DNS. The file /etc/nsswitch.conf decides the order, and this machine says:

TEXT
$ grep hosts /etc/nsswitch.confhosts:          files dns

files means the file /etc/hosts first, then dns. The DNS servers to ask are in /etc/resolv.conf; here:

TEXT
nameserver 8.8.8.8nameserver 8.8.4.4options timeout:2 attempts:3

So the stub resolver asks 8.8.8.8 and waits 2 seconds. If there is no answer, it tries again, up to 3 times, and then tries the second server. A dead first nameserver can therefore cost a long delay on every lookup. (My reading of those two options is from background knowledge; the file itself is real.)

I wanted to prove that "files first" matters. I added 192.0.2.77 hostsonly.example.test to /etc/hosts (and removed it afterwards). Then I asked in different ways:

TEXT
$ dig @127.0.0.1 hostsonly.example.test +short | wc -l0                                  <- dig talks to DNS only, so it sees nothing$ getent hosts hostsonly.example.test192.0.2.77      hostsonly.example.test       <- the system lookup finds it in /etc/hosts

dig skips /etc/hosts completely. So "dig says it does not exist, but my program connects fine" (or the other way round) is a real thing. getent hosts name shows what the operating system lookup returns, and dig shows what DNS says. When they disagree, you have found your problem.

Node.js has two different APIs, and they behave differently. I pointed Node's own resolver at my lab with dns.setServers(["127.0.0.1"]) and tried both functions on two names:

JavaScript
import dns from "node:dns/promises";dns.setServers(["127.0.0.1"]);for (const name of ["hostsonly.example.test", "web.example.test"]) {  for (const fn of ["lookup", "resolve4"]) {    try {      const r = fn === "lookup" ? await dns.lookup(name, { all: true }) : await dns.resolve4(name);      console.log(fn.padEnd(8), name.padEnd(24), JSON.stringify(r));    } catch (e) { console.log(fn.padEnd(8), name.padEnd(24), "FAIL", e.code); }  }}
TEXT
lookup   hostsonly.example.test   [{"address":"192.0.2.77","family":4}]resolve4 hostsonly.example.test   FAIL ENOTFOUNDlookup   web.example.test         FAIL ENOTFOUNDresolve4 web.example.test         ["192.0.2.10","192.0.2.11"]

The result is crossed. dns.lookup (also used by fetch, http and net.connect) calls the operating system, so it sees /etc/hosts but ignores setServers. dns.resolve4 talks DNS itself, so it uses my servers but cannot see /etc/hosts. lookup ran in my sandbox against the system resolver 8.8.8.8, which cannot know my .test names, so it failed for web.example.test. In real code, almost every network call goes through lookup. From background knowledge (I did not read the Node docs for this), lookup runs on the libuv thread pool, which has four threads by default, so a burst of slow lookups can delay unrelated work such as file reads. I did not test that.

A container often has a different /etc/resolv.conf and /etc/hosts from your laptop. "Works on my laptop, ENOTFOUND in Docker" is usually a different resolver, a missing search domain, or an internal name that only the company resolver knows. Check from inside the container: cat /etc/resolv.conf and getent hosts the-name.

Debugging DNS with dig

The commands I used in this lab are the ones I use at work:

CommandWhat it shows
dig nameA normal lookup through your default resolver, with the full reply
dig +short nameOnly the answers, one per line. Good for scripts
dig +noall +answer nameOnly the answer section, with TTLs
dig @1.1.1.1 nameAsk one specific resolver instead of the default
dig @ns1.example.com name +norecAsk an authoritative server directly. The truth, without any cache
dig name TYPEA specific record type: MX, TXT, AAAA, NS, SOA, CNAME
dig +trace nameWalk from the root down, hop by hop
dig -x 192.0.2.10Reverse lookup (PTR). Not tested in my lab
getent hosts nameWhat the operating system (including /etc/hosts) returns

A debugging order that solves most cases:

  1. Is it DNS at all? Try the IP address directly (curl --resolve or http://IP with a Host header). If it works, the problem is the name.
  2. What does the OS say? getent hosts name. If this differs from dig, look at /etc/hosts and nsswitch.conf.
  3. What does the resolver say? dig name and read the status and TTL. A TTL that counts down between two runs means you are reading a cache.
  4. What does the authority say? Find the nameservers with dig NS example.com +short and ask one with dig @ns... name +norec. If it is right there and wrong at the resolver, it is just caching: wait for the TTL. If it is wrong there, fix the zone.
  5. Is the delegation right? dig +trace shows if the parent zone points to the right nameservers. A common mistake after a provider change is that the NS records at the registrar still point to the old DNS host.
  6. Is some nameserver dead or slow? If you see random timeouts, ask each listed NS in turn. One broken server out of three produces "sometimes works".

A short word on security

DNS was designed when the internet was small and trusting. A plain DNS answer over UDP is not signed or encrypted. Three things to know, all from background reading; I did not test any of them here:

  • Spoofing. An attacker who can guess or see the query ID and port can send a fake answer first. The 16 bit ID in the query is small, which is why resolvers also randomise the source port. DNSSEC adds signatures to records so a resolver can check them.
  • Privacy. Anyone on the path can read plain DNS queries and see which names you look up. DNS over HTTPS (DoH) and DNS over TLS (DoT) encrypt the path between you and your resolver. They do not hide the names from the resolver itself.
  • Dangling records. A CNAME that points to a service you no longer use (an old cloud bucket, an old hosting name) can be claimed by someone else, who then serves content on your subdomain. Remove DNS records when you remove the service. See Security Lesson 7 for the server side of this thinking.

Try it

You need dig (package dnsutils or bind-utils) and Python 3.

  1. Run dig +short example.com, then dig example.com and dig +noall +answer example.com a few seconds apart. Does the TTL go down? What does that tell you?
  2. Run dig example.com NS +short. Pick one name server and run dig @that-server example.com +norec. Is the TTL the same each time you ask it? Why or why not?
  3. Run dig +trace example.com. How many hops are there, and which servers answer each hop? (This needs a network with working DNS.)
  4. Run getent hosts localhost and dig +short localhost. Why can they differ?
  5. Add 203.0.113.5 myapp.local to /etc/hosts (use sudo). Run ping -c1 myapp.local, getent hosts myapp.local, and dig +short myapp.local. Which succeed? Remove the line afterwards.
  6. Look up a name that surely does not exist, such as dig this-name-does-not-exist-12345.example.com. What is the status? Run it twice and compare the query times.
  7. Write a Python snippet with socket.getaddrinfo("example.com", 443) and print the results. How many addresses do you get, and of which families?

Answers

  1. Yes, from the second query on, the TTL counts down when you read a cached answer from a recursive resolver. It means you are reading a cache and not the authority. (The exact number depends on the site.)
  2. An authoritative server returns the same TTL each time, because it does not count down; it is the original value. Only caches count down.
  3. Typically four sections: the root servers, the TLD servers (com.), the domain's own nameservers, and the final answer. The exact servers change.
  4. getent uses /etc/hosts and the system rules (files dns), dig asks DNS only. localhost is usually defined in /etc/hosts, so dig may return nothing.
  5. ping and getent succeed (they read /etc/hosts). dig returns nothing, because it does not read that file.
  6. NXDOMAIN. The second query is usually faster, because the "no" is cached for the SOA's negative TTL.
  7. Usually both IPv4 (AF_INET) and IPv6 (AF_INET6) addresses, each repeated for each socket type. On a machine without IPv6 you may see only IPv4. Your code should try them in turn.

Interview questions

What happens when I type www.example.com in the browser, from the DNS point of view? (Must know)

The browser checks its own cache, then asks the operating system, which checks /etc/hosts and its cache, and then sends a question to its configured recursive resolver. If the resolver has no cached answer, it asks a root server, which refers it to the com servers, which refer it to the nameservers of example.com, which give the final answer with a TTL. The resolver caches the answer and returns it. I built this chain in a lab and watched each hop with dig +norec: two referrals and then an answer with the aa flag.

What is the difference between a recursive resolver and an authoritative server? (Must know)

An authoritative server holds the real records for a zone and gives the final answer for those names. It does not hunt for answers about other domains. A recursive resolver does the hunting for the client: it follows referrals from the root down and caches what it learns. In dig output the authoritative answer has the aa flag, and a resolver's answer has ra (recursion available) and a counting-down TTL.

I changed an A record, but some users still reach the old server. Why? (Must know)

Resolvers around the world cached the old answer, and each one keeps it until its TTL runs out. Nothing pushes the change. I tested this: I changed a record with a 20 second TTL, and the resolver kept giving the old IP for the rest of the countdown while the authoritative server already had the new one. I check by asking the authoritative server directly and the user's resolver separately, and I wait for the TTL. For future changes I lower the TTL a few days before the move.

How do you move a site to a new IP with minimal downtime? (Senior signal)

I lower the TTL (for example to 60 seconds) well ahead, at least as long as the old TTL. Then I bring up the new server and test it directly. Then I change the record. I keep the old server running until the old TTL, and the traffic, have drained. Then I raise the TTL again. Also I check that the new server is correct before the change, because rolling back is also limited by TTL.

What do NXDOMAIN, NODATA and SERVFAIL mean? (Must know)

NXDOMAIN says the name does not exist. NODATA is a NOERROR answer with nothing in it: the name exists but not for the type asked, for example an AAAA query for a name with only an A record. SERVFAIL says the resolver could not get an answer, because an authoritative server was down, broken, or (with DNSSEC) its answer was invalid. In the lab I produced all three. NXDOMAIN and NODATA are normal answers that are cached. SERVFAIL points to a problem on the way to the authority.

What is negative caching, and how can it hurt? (Senior signal)

Resolvers also cache "this name does not exist", for a time that comes from the zone's SOA record. In my test the second query for a missing name came from cache, with a query time of 0 ms. It hurts when someone looks up a name before it is created, for example a new subdomain, a CI check or a health probe. The "no" is then remembered, and users keep getting NXDOMAIN for some time after I add the record. I create the record before anything queries it, or I wait out the negative TTL.

Why can't I put a CNAME at the root of my domain (example.com)? (Senior signal)

A CNAME means "this name is an alias for another name, and has no other records". But the root of the zone (the apex) must hold SOA and NS records, and usually MX and TXT too, so a CNAME cannot live there. That is the rule from the DNS standards, which I know from background and did not test. Providers offer ALIAS or ANAME records, or "CNAME flattening", which return the final A records for the apex while keeping the zone valid. For www, a normal CNAME is fine; my lab uses www as a CNAME to web.

Why does dig say the name does not exist while my program can connect to it? (Trap)

dig asks DNS only. My program uses the operating system lookup, which reads /etc/hosts first (the files dns order in nsswitch.conf), and may use other sources too. I proved it by adding a name to /etc/hosts: dig returned nothing and getent hosts found it. The opposite also happens: a stale /etc/hosts line can override correct DNS. getent hosts shows what programs see, and dig shows what DNS says.

What is the difference between dns.lookup and dns.resolve in Node.js? (Senior signal)

dns.lookup calls the operating system function, so it respects /etc/hosts and the system settings. It is what http, fetch and net.connect use. dns.resolve* talks to DNS servers directly with Node's own resolver, so it ignores /etc/hosts and can use dns.setServers. My test with both functions and two names gave crossed results, which proves they take different paths. In production I use lookup through the normal HTTP clients, and I remember that lookup uses the thread pool, so a flood of slow lookups can slow other work.

Does DNS use UDP or TCP? (Trap)

Both. Most queries are one small UDP packet to port 53 with one UDP reply. If the answer does not fit, the server sets the tc (truncated) flag, and the client repeats the question over TCP. I captured this: a UDP question, a 34 byte truncated reply, then a TCP handshake and the 1050 byte answer. TCP is also used for zone transfers between servers. So firewalls must allow both UDP and TCP on port 53.

Users in one office cannot reach the site, others can. How do you check whether it is DNS? (Senior signal)

I compare what the two places get. From the failing office I run dig name and getent hosts name, and read the status, the answer and the TTL. I also run dig @8.8.8.8 name to compare with a public resolver, and dig @authoritative-ns name +norec for the truth. If the office resolver returns something different, it is a cache, a split-horizon (internal) zone, or a filter. If the answers agree, I test the connection to the IP, and then the problem is not DNS, it is routing or a firewall. The trap is trusting ping or the browser instead of looking at the actual answers.

What can go wrong if I resolve a hostname once and keep the IP? (Trap)

The IP may change: a failover, a rolling deployment, a cloud load balancer that rotates addresses. My process would keep sending traffic to the old address, which may be gone or belong to someone else. DNS answers come with a TTL for exactly this reason. I should resolve for each new connection or respect the TTL, and let connection pools drop old connections so new ones get a fresh lookup. Some runtimes (the JVM, for one) have their own DNS cache setting, so I check the runtime too.

Key takeaways

  • DNS is a tree. The root points to the TLD servers, those point to the domain's authoritative servers, which give the answer. A recursive resolver walks the tree for you and caches the result.
  • A referral has no aa flag. A final answer from the authority does. dig @server name +norec asks one server one question and shows what it knows.
  • Every record has a TTL, and caches count it down. Changes are not pushed. Old answers expire. Lower the TTL before a move.
  • Negative answers (NXDOMAIN) are cached too, for the SOA's last number.
  • Record types: A and AAAA (addresses), CNAME (alias, not at the zone apex), MX (mail), TXT (text, SPF, verification), NS (nameservers), SOA (zone info).
  • NOERROR with an empty answer is NODATA, not a failure. SERVFAIL means the resolver could not get a good answer.
  • DNS runs on UDP port 53 with a tiny binary format, and switches to TCP when an answer is truncated. Allow both in firewalls.
  • Your program uses the OS lookup (/etc/hosts first). dig uses DNS only. If they disagree, use getent hosts name to see the program's view.
  • Debug in order: IP works? OS answer? resolver answer? authority answer? delegation? Then decide between caching and a real mistake.

What's next

Next: HTTP/1.1, HTTP/2 and HTTP/3. Now that you know how a name becomes an address (this lesson) and how a connection is opened and kept honest (lesson 2), we can look at the conversation inside the connection. You will read real HTTP/1.1 requests on the wire, see why keep-alive and connection reuse matter, and compare HTTP/2 multiplexing and HTTP/3 over QUIC (which runs on UDP). Practise now: run dig +trace and dig +noall +answer on a site you own, and write down the TTL of each record. Would you be happy with that TTL if you had to move the site tomorrow?

Published by Jainik · Last updated · How these lessons are made

Keep learning

Related lessons

Part of the study guide

Related series