Service Discovery: How Services Find Each Other

The service registry

Service registry
┌──────────────────┬────────────────────┬──────────┐
│ service          │ instance address   │ health   │
├──────────────────┼────────────────────┼──────────┤
│ payment-service  │ 10.0.4.17:8080     │ ✅       │
│ payment-service  │ 10.0.7.23:8080     │ ✅       │
│ payment-service  │ 10.0.2.91:8080     │ ❌ failing│
│ order-service    │ 10.0.5.44:8080     │ ✅       │
└──────────────────┴────────────────────┴──────────┘

The registry must be highly available, which means it must almost always be up. If it is down, nobody can find anybody. So registries usually run as a cluster (a group of servers that keep copies of the same data). They often use consensus, a way for servers to agree on one answer. Raft is a common consensus algorithm (post 24).

Popular registries:

• Consul (HashiCorp): service registry, health checks, DNS interface and key-value store.
• etcd: a key-value store (a simple database of names and values) that always gives correct data. Kubernetes keeps its state in it.
• Apache ZooKeeper: an older coordination service, used by many big-data systems.
• Netflix Eureka: built for AWS. It favours availability over consistency, which means it keeps answering even if its data may be slightly old.
• Kubernetes' built-in discovery (Services + DNS), the most common today. Kubernetes is a system that runs and manages containers. DNS is the system that turns names into IP addresses.
• Cloud services: AWS Cloud Map, and similar.

How instances get registered

Self-registration. Each instance adds itself to the registry when it starts. It sends heartbeats (a small "I am alive" message every few seconds) and removes itself when it stops.

• ✅ Simple.
• ❌ Every service needs registry code, and a crashed instance cannot remove itself. A heartbeat timeout handles that. A TTL (time to live) is the time after which an entry with no heartbeat is deleted.

Third-party registration. The platform registers instances for you. For example, Kubernetes knows exactly which pods are running and ready. A pod is the unit that runs your containers. Kubernetes updates the service's list of addresses (its endpoints) by itself.

• ✅ No code in the services, and always accurate.
• This is how most modern platforms work.

Health checks

Only healthy instances should receive traffic:

• the registry or platform polls a health endpoint. This means it calls a URL like /health or /ready again and again to ask "are you OK?",
• or instances send heartbeats within a TTL,
• instances that fail are removed from the list until they recover.

As in post 12, keep health checks focused on the instance itself. A deep check also tests shared dependencies, like the database. If that dependency fails for a moment, such a check can remove every instance at once.

Client-side discovery

The caller asks the registry for the list of instances and picks one itself. Load balancing means spreading requests across instances, and here the client does it:

• ✅ No extra hop (no extra server in the path), and the client can use smart balancing, like choosing the instance with the fewest requests or the one in the same zone.
• ✅ Works well with gRPC (a fast way for services to call each other, post 31). gRPC keeps one connection open for a long time, so it needs balancing for each request.
• ❌ Every client needs discovery and balancing code, in every programming language you use.
• Examples: Netflix's Eureka + Ribbon setup (Ribbon is now in maintenance mode), and the name resolvers and balancers built into gRPC.

Server-side discovery

The caller sends requests to a load balancer or router that has a fixed address. The router looks up the registry and passes the request on:

• ✅ Clients stay simple. They only call one address.
• ✅ Centralised control of balancing and routing.
• ❌ An extra network hop, and the router must itself be highly available.
• Examples: AWS load balancers with target groups (lists of instances), and Kubernetes Services. In Kubernetes, kube-proxy sends traffic for a virtual IP (an address that belongs to no single machine) to the pods.

DNS-based discovery

This is the simplest way: you look up the service by name through DNS and get back IP addresses. DNS (Domain Name System) is the system that turns names into IP addresses.

payment-service.payments.svc.cluster.local  →  10.96.12.40 (Kubernetes Service IP)

In Kubernetes:

• a Service gives a group of pods a name and a virtual IP that do not change,
• cluster DNS (CoreDNS, the DNS server inside Kubernetes) turns payment-service into an address when you are in the same namespace (a named group of resources). It also understands the full name payment-service.payments.svc.cluster.local,
• traffic to the Service IP is spread across the pods that are ready,
• "headless" Services (Services without a virtual IP) return the IP of each pod. They are used for client-side balancing or for stateful apps (apps that keep data, like databases).

DNS pitfalls (post 1):

• Caching: clients and language runtimes may keep (cache) DNS answers longer than you expect, and keep calling old instances that are already gone. Keep TTLs low, and check how your runtime caches DNS.
• DNS gives addresses, not health or load. It relies on the platform keeping records up to date.
• SRV records are a kind of DNS record that can also include a port number, but not every client supports them.

Service mesh discovery

A service mesh (Istio, Linkerd, Consul) is a layer that manages the traffic between services. In a mesh, a sidecar proxy runs next to each service. A sidecar is a small helper program, and it is often Envoy, a popular proxy. The mesh's control plane is the part that manages all the sidecars. It watches the registry (usually Kubernetes) and sends up-to-date address lists to every sidecar. Envoy calls these APIs xDS.

Your application code just calls http://payment-service. The sidecar handles discovery, load balancing, retries, timeouts, circuit breaking (stopping calls to a failing service), mTLS (both sides prove their identity, post 51) and metrics (post 46).

Making deploys and shutdowns safe

Sometimes an instance must be removed, for example during a deploy, when scaling down, or when a node (a machine) is drained. Do these steps:

1. Mark it not-ready, so it's removed from discovery and load balancers first,
2. wait for discovery to update everywhere (a few seconds),
3. drain the instance. This means you let the requests that are still running (in-flight requests) and the open connections finish,
4. then stop the process.

In Kubernetes, you use readiness probes (a check that tells Kubernetes if a pod can take traffic), a preStop hook (a command that runs just before the pod stops) with a short sleep, and a sensible terminationGracePeriodSeconds (the maximum time a pod gets to stop). Without these, callers keep sending requests to a pod that is already shutting down, and users see errors on every deploy.