Docker & Kubernetes
Services, Ingress and Networking in Kubernetes
How pods find each other, ClusterIP, NodePort and LoadBalancer, DNS names, Ingress, and what a service mesh is for.
On this page
- Words used in this lesson
- The problem: pods have short lives and moving addresses
- Labels and selectors: how Kubernetes "points" at pods
- Your first Service: ClusterIP
- How the Service finds its pods: EndpointSlices
- Service types
- DNS names: how a pod finds a Service
- What really happens: kube-proxy and the virtual IP
- Ingress: HTTP routes from outside
- Reaching a Service from your laptop
- NetworkPolicy: who may talk to whom
- What a service mesh is for (and why you probably do not need one yet)
- A real mini lab you can do without a cluster
- Commands to know (reviewed, not run on a cluster here)
- Try it
- Interview questions
- Key takeaways
- What's next
In one line: pods come and go and their IP addresses change, so Kubernetes gives you a Service, a stable name and address that always points at whichever pods are healthy right now, and an Ingress (or its newer sibling, the Gateway API) to let HTTP traffic from the outside world reach those Services.
In the last lesson you made a Deployment. It keeps three copies of your app running. That is great, but there is a problem you have probably already seen: if the pods can be killed and replaced at any time, how does anything find them? You cannot write http://10.244.1.7:3000 in your code, because that pod may be gone in ten minutes and its replacement will have a different address.
This lesson solves that problem step by step. First we meet the Service, which is the answer inside the cluster. Then we meet the Ingress, which is the answer from outside the cluster. Along the way we look at DNS names, the four Service types, headless Services, network policies, and what a "service mesh" is for.
This is lesson 5 of 9. It builds on Kubernetes Core Ideas: Cluster, Pods and Deployments, where we made the notes-api Deployment. The next lesson, Configuration, Secrets and Storage, shows how to give those pods settings and disk space. If you want the Docker-only version of networking first, Volumes, Networks and Running Containers Safely covers ports and user-defined networks.
What I ran / checked. Tools on my machine: Docker Engine 29.8.2 (a real daemon, so the Docker examples below are real runs), kubectl v1.37.1 (client only), kubeconform v0.8.0 (checks YAML against the official Kubernetes JSON schemas, here for Kubernetes 1.34.0), yamllint 1.38.0 and kind v0.33.0. I could not run a real Kubernetes cluster. I pulled the kind node image (kindest/node:v1.34.0) and tried to create a cluster. kind's control plane never became healthy, because the nested container runtime inside the sandbox could not start the control plane pods (the kubelet reported a runc create failed ... can't get final child's PID from pipe: EOF error). So every Kubernetes manifest here is labelled "reviewed, not run on a cluster here". What I did do for real: every manifest passed kubeconform -strict against the 1.34.0 schemas and yamllint, I made kubectl create ... --dry-run=client -o yaml print the YAML it would send, and I deliberately broke manifests to see the real error text. Anything that only a cluster can show (events, pod IPs, kubectl get endpointslices output) I describe in words, and I do not paste fake terminal output. Sources I read (own words below): the Kubernetes website docs on Service, Ingress, Ingress controllers, Gateway API, DNS for Services and Pods, and Network Policies (raw markdown from the kubernetes/website repository on GitHub), plus the official JSON schemas used by kubeconform. Could not reach: the rendered kubernetes.io site, registry.k8s.io and Docker Hub (rate limited), so images came from mirror.gcr.io. I did not read any service mesh project's own documentation, so the mesh section is short and general. I verified no GitHub star counts.
Words used in this lesson
- Pod IP is the address a single pod gets inside the cluster. It changes when the pod is replaced.
- Label is a small key and value tag on any Kubernetes object, such as
app: notes-api. - Selector is a rule that picks objects by their labels, such as "every pod with
app: notes-api". - Service is a stable name and virtual address in front of a changing set of pods.
- ClusterIP is the default Service type. The address works only inside the cluster.
- NodePort is a Service type that also opens one port on every node.
- LoadBalancer is a Service type that asks the cloud for an external load balancer.
- Headless Service is a Service with no virtual address. DNS returns the pod addresses directly.
- EndpointSlice is the list of pod addresses (and whether each is ready) behind a Service.
- kube-proxy is the node component that makes a Service address actually work.
- Cluster DNS is the built-in DNS server that turns Service names into addresses.
- Ingress is an object that describes HTTP and HTTPS routes from outside the cluster to Services.
- Ingress controller is the program that reads Ingress objects and does the real routing. Without one, an Ingress does nothing.
- Gateway API is the newer, richer set of objects for routing traffic into a cluster (GatewayClass, Gateway, HTTPRoute).
- NetworkPolicy is an object that says which pods may talk to which pods.
- Service mesh is extra infrastructure that adds things like encryption and retries between services.
The problem: pods have short lives and moving addresses
Every pod in a cluster gets its own IP address. Any pod can talk to any other pod using that address, without NAT. That is part of the Kubernetes networking model. So in theory you could just ask for the IP of your database pod and connect.
In practice this breaks at once, for three reasons:
- Pods are replaced, not repaired. When a pod dies, the Deployment creates a new pod. The new pod has a new IP.
- Scaling changes the set. Today you have three API pods. Tomorrow you have ten. A client should not need to know.
- Rolling updates replace every pod. During a rollout all the old addresses disappear.
If you used Docker Compose in lesson 3, you already saw the fix for the same problem on one machine: you connect to db, not to an IP, and Docker's DNS finds it. Kubernetes does the same, with more rules. The object that does it is the Service.
Labels and selectors: how Kubernetes "points" at pods
Before the Service, one idea you must be comfortable with: labels and selectors. Kubernetes almost never says "this exact pod". It says "all pods that look like this".
A label is a key and value you put on an object. In our Deployment from lesson 4, the pod template has:
template: metadata: labels: app: notes-apiEvery pod that Deployment makes carries app: notes-api. A selector is a question about labels. app: notes-api means "give me everything with that label and that value". The Deployment itself uses a selector (spec.selector.matchLabels) to know which pods belong to it. A Service uses exactly the same trick to know which pods to send traffic to.
Here is the Deployment again, so this lesson stands alone. It is the same shape as lesson 4, with a named port.
# deployment.yamlapiVersion: apps/v1kind: Deploymentmetadata: name: notes-apispec: replicas: 3 selector: matchLabels: app: notes-api template: metadata: labels: app: notes-api spec: containers: - name: api image: notes-api:1.0 ports: - name: http containerPort: 3000notes-api:1.0 is a placeholder for the image you built in lesson 1 and 2 (it would have to be in a registry your cluster can reach). Notice the port has a name, http. That small detail will pay off in a minute.
Status: reviewed, not run on a cluster here. This file passed kubeconform -strict -kubernetes-version 1.34.0 and yamllint. I could not apply it to a real cluster.
Your first Service: ClusterIP
This is the smallest useful Service.
# svc-clusterip.yamlapiVersion: v1kind: Servicemetadata: name: notes-apispec: selector: app: notes-api ports: - name: http port: 80 targetPort: httpRead it line by line:
metadata.name: notes-apiis the name other pods will use. It becomes a DNS name (more on that soon).spec.selectorsays: send traffic to pods labelledapp: notes-api.port: 80is the port the Service listens on. Clients connect tonotes-api:80.targetPort: httpis the port on the pod where traffic goes. Because we gave the container port the namehttp, we can use the name instead of the number3000.- No
typemeansClusterIP, the default.
Status: reviewed, not run on a cluster here. Validated with kubeconform -strict and yamllint. kubectl apply was not possible.
The reason to use a named targetPort is that you can change the container's number later (say from 3000 to 8080) in one place, the Deployment, and the Service keeps working. With a number you must change two files and hope you did not forget one. The Kubernetes docs list this as a benefit of named target ports.
The four port numbers, in one table
The most common beginner confusion is that "port" appears in four places. Here is the whole map:
| Field | Where it lives | What it means |
|---|---|---|
containerPort | In the pod spec | The port your app listens on inside the container. Mostly documentation, it does not open anything. |
targetPort | In the Service | Which pod port the Service sends traffic to. Number or name. |
port | In the Service | The port the Service itself listens on. What clients use. |
nodePort | In a NodePort Service | A port opened on every node (default range 30000 to 32767). |
If targetPort is left out, Kubernetes uses the same number as port. That is a classic trap: you set port: 80, forget targetPort, and the Service sends traffic to port 80 of a pod that listens on 3000. The connection is refused and nothing in the YAML looks wrong.
What you get from a ClusterIP
When the Service is created, Kubernetes gives it a virtual IP (the cluster IP) that does not change for the life of the Service. Cluster DNS also gets a name for it. A pod that connects to http://notes-api (from the same namespace) reaches one of the three API pods.
Two important things about this address:
- It is virtual. No network card has this IP. Nothing "listens" on it. The node's networking rules (set up by kube-proxy, see below) rewrite traffic aimed at it so it lands on a real pod.
- It only works inside the cluster. That is what the name means. From your laptop you cannot reach a ClusterIP. We will fix that in the Ingress and port-forward parts.
How the Service finds its pods: EndpointSlices
You wrote a selector. Who turns that into a list of addresses? A control loop in the control plane. It watches pods and Services, and for each Service it keeps an EndpointSlice object up to date. An EndpointSlice is a list of pod IPs and ports behind a Service, plus a flag for each address that says whether the pod is ready.
Some facts straight from the docs:
- EndpointSlices are the current API. The older
Endpointsobject is deprecated (since v1.33). One reason: it cuts off the list when it gets too long for one object. - By default, a new EndpointSlice is started once the existing ones each hold at least 100 endpoints. So big Services are split into slices.
- Only ready pods receive traffic. This is why readiness probes matter (lesson 7). A pod that is running but not ready is in the list, but marked not ready, and clients are not sent to it.
On a real cluster you would look at this with kubectl get endpointslices or kubectl describe service notes-api. I could not run those here, so I will not show output. What to look for is this: if the list of endpoints is empty, no pod matched the selector (or none is ready), and every request to the Service will fail. This is the single most useful check when "the Service doesn't work", and we will use it again in lesson 8.
Here is a Service with a typo in the selector. It is perfectly valid YAML and passes schema checks, which is exactly why it is dangerous:
# svc-mismatch.yamlapiVersion: v1kind: Servicemetadata: name: notes-apispec: selector: app: notes-apii # extra "i": matches no pod ports: - port: 80 targetPort: 3000$ kubeconform -strict -summary -kubernetes-version 1.34.0 svc-mismatch.yamlSummary: 1 resource found in 1 file - Valid: 1, Invalid: 0, Errors: 0, Skipped: 0(That output is real.) The schema checker cannot know that no pod has the label app: notes-apii. Only the cluster knows. The result in a real cluster would be a Service with an empty endpoint list. The lesson: a green YAML check does not mean a working app.
Service types
The type field decides how far the Service is reachable. There are four types you should know.
| Type | Reachable from | How | Typical use |
|---|---|---|---|
ClusterIP (default) | Inside the cluster only | Virtual IP and DNS name | Service-to-service calls, databases |
NodePort | Outside, via any node's IP | Also opens one port on every node | Quick tests, or as a building block for a load balancer |
LoadBalancer | Outside, via an external address | Asks the cloud provider to create a load balancer | Exposing one service directly in the cloud |
ExternalName | Inside the cluster | DNS alias (a CNAME) to an outside name | Giving a friendly in-cluster name to an outside service |
Each builds on the one before it. The docs describe NodePort as "ClusterIP plus a port on every node", and LoadBalancer as "NodePort plus an external load balancer" (on most cloud setups).
NodePort
# svc-nodeport.yamlapiVersion: v1kind: Servicemetadata: name: notes-api-nodeportspec: type: NodePort selector: app: notes-api ports: - port: 80 targetPort: http nodePort: 30080Now http://<any-node-ip>:30080 reaches the API. If you leave nodePort out, Kubernetes picks a free one from the range 30000 to 32767 (the default of the --service-node-port-range flag).
Status: reviewed, not run on a cluster here. Passed kubeconform -strict and yamllint.
Why not use NodePort for production? Because clients must know a node's IP, nodes come and go, and you are exposing a high, odd port on every machine. It is fine for a lab or for a load balancer to sit in front of.
LoadBalancer
# svc-lb.yamlapiVersion: v1kind: Servicemetadata: name: notes-api-publicspec: type: LoadBalancer selector: app: notes-api ports: - port: 80 targetPort: httpOn a cloud provider (or with a controller that supports it) this creates a real external load balancer and writes its address into the Service status. On a laptop cluster with no cloud, the external address often stays <pending> forever, because nothing is there to create it. The docs say the actual load balancing is done by the cloud provider's own system. You pay for each one, so one LoadBalancer per service gets expensive fast. That cost is the main reason Ingress exists.
Status: reviewed, not run on a cluster here. Passed kubeconform -strict and yamllint.
ExternalName
An ExternalName Service has no selector and no pods. It makes cluster DNS return a CNAME to a name you give it, for example db.example.com. It is a naming trick, not a proxy: it does no port changes and no load balancing, and with HTTPS the host name the client sees might not match the certificate. Use it rarely.
Headless Services: no virtual IP at all
Sometimes you do not want one virtual address. You want to know every pod, for example for a database cluster where each member must talk to specific members. Setting clusterIP: None makes a headless Service:
# svc-headless.yamlapiVersion: v1kind: Servicemetadata: name: notes-api-podsspec: clusterIP: None selector: app: notes-api ports: - port: 3000The docs say: for a headless Service, no cluster IP is allocated, kube-proxy does not handle it, and there is no load balancing done by the platform. Instead, cluster DNS answers a lookup of the Service name with the addresses of all ready pods. The client chooses. StatefulSets (lesson 6) depend on this.
Status: reviewed, not run on a cluster here. Passed kubeconform -strict and yamllint.
Note that clusterIP: None is a special string. The docs warn that it is not the same as leaving the field out.
I cannot run cluster DNS here, but Docker has the same idea in a smaller form, and I can show it for real. If several containers on a user-defined network share one network alias, Docker's built-in DNS returns all of their addresses:
docker network create demo-netfor n in 1 2 3; do docker run -d --rm --name web$n --network demo-net --network-alias web \ mirror.gcr.io/library/busybox:1.36 sh -c "echo web$n > /i; httpd -f -p 8080 -h /"donedocker run --rm --network demo-net mirror.gcr.io/library/busybox:1.36 sh -c \ 'nslookup web 2>&1 | tail -8; for i in 1 2 3 4 5 6; do wget -qO- http://web:8080/i; done'Non-authoritative answer:Name: webAddress: 172.19.0.2Name: webAddress: 172.19.0.4Name: webAddress: 172.19.0.3web3web1web1web1web1web3One name, three addresses, and the client ended up at different containers (the order and mix change from run to run). This is client-side choice over DNS, the same idea as a headless Service. It also shows the weakness: the client picked web1 four times out of six. DNS does not balance load evenly, and clients may cache the answer. A normal ClusterIP Service does the balancing on the node instead.
DNS names: how a pod finds a Service
Cluster DNS gives every Service a name. The full form is:
<service>.<namespace>.svc.cluster.localSo our Service in the default namespace is notes-api.default.svc.cluster.local. You do not have to type all that. Each pod gets a /etc/resolv.conf with a search list. The docs show it like this (for a pod in some namespace):
search <namespace>.svc.cluster.local svc.cluster.local cluster.localoptions ndots:5Because of the search list:
- From a pod in the same namespace,
notes-apiis enough. - From a pod in another namespace, use
notes-api.default(the namespace is part of the name). - The long form always works.
The ndots:5 option means: if a name has fewer than five dots, try the search list first. This is helpful, but it has a side effect: a lookup for an outside name like api.stripe.com (two dots) may first try several wrong names such as api.stripe.com.default.svc.cluster.local. That is extra DNS traffic. If you see lots of DNS lookups for outside names, adding a trailing dot (api.stripe.com.) makes the name absolute. I name this as a known detail of the setup; I did not measure its cost.
Named ports also get SRV records, which return both the address and the port number. You will rarely use them by hand.
Do not use IP addresses of Services or pods in config files. Use the DNS name. The Service IP is stable for the Service's life, but if you delete and recreate the Service, it may change. The name never does.
The older way: environment variables
Kubernetes also injects environment variables like NOTES_API_SERVICE_HOST into pods. The docs mention them as the second discovery method after DNS. They have a catch: they are set when the pod starts, so a Service created after the pod is invisible to it. DNS has no such problem. Prefer DNS.
What really happens: kube-proxy and the virtual IP
A Service IP that no machine owns sounds like magic, so let us take the magic out. Each node runs a component called kube-proxy. It watches Services and EndpointSlices and programs the node's own packet rules (iptables, IPVS or nftables, depending on the setup) so that packets sent to the Service IP and port are rewritten to the IP and port of one chosen ready pod. The choice happens in the kernel, on the node where the client pod runs.
Two consequences are worth remembering:
- There is no central proxy server for ClusterIP traffic. Each node does the rewriting for its own pods. That is why it scales well and why you cannot
pinga Service IP: ping uses a protocol the rules do not rewrite. - Headless Services skip all this, as the docs say. DNS gives the real pod IPs.
The docs also describe knobs that change which backends are chosen: internalTrafficPolicy and externalTrafficPolicy (for example, only use pods on the same node), trafficDistribution (prefer pods in the same zone), and session affinity by client IP. They are advanced. The key point for now: by default the spread is simple and not smart. It does not look at how busy each pod is.
A Service is a connection-level load balancer, not an HTTP one. It picks a pod when a connection opens. With long-lived connections (gRPC, HTTP/2, WebSockets), one client can keep talking to one pod for hours, so a new pod you scaled up may get no traffic from existing clients. For those protocols you may need a smarter proxy, a mesh, or client-side balancing.
Ingress: HTTP routes from outside
Now the outside world. One LoadBalancer per service is expensive and clumsy: you get one IP per service, no host names, no paths, no TLS handling. Ingress solves this for HTTP and HTTPS. It is a set of rules: "requests for host X and path Y go to Service Z". One entry point, many Services behind it.
Here is the key idea, which trips up many people: an Ingress is only a set of rules, written as a Kubernetes object. It does nothing by itself. You must run an Ingress controller, a program that reads Ingress objects and configures a real proxy. The docs say it plainly: only creating an Ingress resource has no effect. Common controllers are proxies like NGINX, HAProxy, Traefik and cloud load balancers. Each one behaves slightly differently, and the docs warn you to read the controller's own documentation.
Here is a complete Ingress:
# ingress.yamlapiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: notesspec: ingressClassName: nginx tls: - hosts: - notes.example.com secretName: notes-tls rules: - host: notes.example.com http: paths: - path: /api pathType: Prefix backend: service: name: notes-api port: number: 80 - path: / pathType: Prefix backend: service: name: notes-web port: number: 80Status: reviewed, not run on a cluster here. Passed kubeconform -strict and yamllint. It assumes a controller whose class is named nginx, a Service called notes-web, and a TLS Secret named notes-tls that I did not create.
How to read it:
ingressClassName: nginxsays which controller should handle this Ingress. If a cluster has several controllers, this chooses. If you leave it out, the cluster needs a default IngressClass, or no controller may pick it up. The docs recommend always defining a default class.rules[].hostis the host name. Both host and path must match before traffic is sent on.- Each
pathhas abackendthat names a Service and port, not a pod. Ingress sits on top of Services. tlssays: for these hosts, terminate HTTPS using the certificate in the Secretnotes-tls. The controller does the TLS work, so your pods can speak plain HTTP inside the cluster.
You can also ask kubectl to write the skeleton for you. This is real output from kubectl create ingress with --dry-run=client:
$ kubectl create ingress web --rule="notes.example.com/api*=notes-api:80" --class=nginx --dry-run=client -o yamlapiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: webspec: ingressClassName: nginx rules: - host: notes.example.com http: paths: - backend: service: name: notes-api port: number: 80 path: /api pathType: Prefixstatus: loadBalancer: {}Notice that the trailing * in the rule turned into pathType: Prefix. That is a handy way to avoid hand-typing nested YAML.
pathType: the rule people get wrong
pathType is required in networking.k8s.io/v1. If you forget it, the object is rejected. Here is the real schema error from my test, with a missing pathType and a port number written as a string ("80"):
$ kubeconform -strict -summary -kubernetes-version 1.34.0 ing-nopath.yamling-nopath.yaml - Ingress bad is invalid: ... at '/spec/rules/0/http/paths/0': missing property 'pathType' - at '/spec/rules/0/http/paths/0/backend/service/port/number': got string, want null or integerSummary: 1 resource found in 1 file - Valid: 0, Invalid: 1, Errors: 0, Skipped: 0(I shortened the line. The two errors are real and both were found in one run.)
There are three path types:
| pathType | Meaning |
|---|---|
Exact | Matches the URL path exactly, and is case sensitive. |
Prefix | Matches by path prefix, split on /, element by element. |
ImplementationSpecific | Up to the controller. It may treat it like Prefix, Exact, or something else. |
Prefix works on whole path elements, not on characters. A few examples taken from the docs table: a Prefix of /foo matches /foo and /foo/. A Prefix of /aaa/bb does not match /aaa/bbb, because bb and bbb are different elements. A Prefix of / matches every path. When more than one rule matches, the longest match wins, and if equal, Exact wins over Prefix.
Ingress annotations are not portable. Features like rewrites, rate limits, timeouts and body size limits are set with annotations that belong to one controller. The same YAML on a different controller silently ignores them. This is one of the reasons the Kubernetes project now points people toward Gateway API.
Ingress is frozen: Gateway API is the future
This is a recent change worth knowing for interviews. The current Kubernetes docs put a note at the top of the Ingress page: the project recommends using Gateway instead of Ingress, and the Ingress API has been frozen. That means:
- Ingress is generally available and stable, with the usual stability promises.
- The project has no plans to remove it.
- But it will get no new features or changes.
So Ingress is not going away, and you will see it in almost every existing cluster. New designs, though, should look at the Gateway API.
Gateway API is a family of resource kinds, installed as an add-on (custom resources), not built into the core API. The docs list four stable kinds:
| Kind | What it is |
|---|---|
GatewayClass | A type of gateway, managed by a controller. Like IngressClass. |
Gateway | An actual instance of traffic-handling infrastructure, such as a cloud load balancer. |
HTTPRoute | HTTP routing rules from a Gateway to backends (usually Services). |
GRPCRoute | The same for gRPC. |
The design goals in the docs: role-oriented (infrastructure team owns the Gateway, app team owns the routes), portable (the same objects across implementations), expressive (header matching and traffic weights without custom annotations) and extensible. Here is the route that matches our Ingress /api rule:
# gateway.yamlapiVersion: gateway.networking.k8s.io/v1kind: HTTPRoutemetadata: name: notesspec: parentRefs: - name: shared-gateway hostnames: - notes.example.com rules: - matches: - path: type: PathPrefix value: /api backendRefs: - name: notes-api port: 80Status: reviewed, not run on a cluster here. This is not a core type, so I checked it with kubeconform using the community CRD schema catalogue (datreeio/CRDs-catalog) for HTTPRoute. It passed. It assumes a Gateway called shared-gateway already exists and that the Gateway API CRDs and a controller are installed. Neither exists in my sandbox.
The point of the split is ownership. A platform team creates the Gateway once (ports, TLS, which namespaces may attach). Each app team writes only its own HTTPRoute, in its own namespace. With Ingress, one object held both, and everyone fought over it.
Reaching a Service from your laptop
You are developing and want to hit a ClusterIP Service without any ingress. kubectl port-forward opens a tunnel from your machine to a Service or pod through the API server:
kubectl port-forward service/notes-api 8080:80# then in another terminal:curl http://localhost:8080/(Reviewed, not run on a cluster here.) It is a debugging tool, one connection path from you to the cluster, not a way to serve users. It also picks one pod and sticks with it, so do not use it to test load balancing. The other ways to reach a Service from outside are the ones above: NodePort, LoadBalancer, Ingress or a Gateway.
NetworkPolicy: who may talk to whom
By default, all pods can talk to all pods. That is convenient and a little scary. A NetworkPolicy lets you restrict this, like a firewall written in labels. Here is a policy that only allows pods labelled app: notes-api to reach the Postgres pods on port 5432:
# networkpolicy.yamlapiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata: name: db-only-from-apispec: podSelector: matchLabels: app: postgres policyTypes: - Ingress ingress: - from: - podSelector: matchLabels: app: notes-api ports: - protocol: TCP port: 5432Status: reviewed, not run on a cluster here. Passed kubeconform -strict and yamllint.
Two things to remember:
podSelectorat the top says which pods the policy protects. Once any policy selects a pod for ingress, traffic to it is denied unless some policy allows it. Thefromlist says who is allowed.- A NetworkPolicy object is only a request. It is enforced by the cluster's network plugin (the CNI). If your CNI does not support policies, you can create the object and nothing happens. This is a real and quiet trap: your database looks protected and is not. The docs say network policies are implemented by the network plugin, so check that yours does.
This is the Kubernetes version of "least privilege" from the Security series: if one pod is compromised, it should not be able to scan and connect to everything else.
What a service mesh is for (and why you probably do not need one yet)
A service mesh puts a small proxy next to each pod (or on each node) and manages the traffic between services. What it can add, in general terms:
- Encryption between pods (mutual TLS), so internal traffic is not plain text.
- Retries, timeouts and circuit breaking configured in one place instead of in every service.
- Traffic splitting, for example send 5 percent of requests to a new version.
- Detailed metrics and traces for every call between services.
The price is real: more moving parts, more memory and CPU, more to learn and debug, and one more thing that can break at 3 a.m. For a handful of services a mesh is usually too much. Retries and timeouts in your own code, plus NetworkPolicies and a normal Ingress or Gateway, cover most small systems. I did not read any mesh's documentation for this lesson, so treat this section as an overview and read the project docs before you choose one.
A real mini lab you can do without a cluster
If you do not have a cluster, you can still practise the habits of this lesson:
- Write the Deployment, Service and Ingress files above.
- Run
kubeconform -strict -summary -kubernetes-version 1.34.0 *.yamlto check the shapes. It downloads the official schemas on first use. - Break the files on purpose and read the errors. Here are two real ones. This Service has two typos,
selecterandtargetport:
$ kubeconform -strict -summary -kubernetes-version 1.34.0 svc-typo.yamlsvc-typo.yaml - Service bad is invalid: ... at '/spec/ports/0': additional properties 'targetport' not allowed - at '/spec': additional properties 'selecter' not allowedWithout -strict, the same file passes (I ran both):
$ kubeconform -summary -kubernetes-version 1.34.0 svc-typo.yamlSummary: 1 resource found in 1 file - Valid: 1, Invalid: 0, Errors: 0, Skipped: 0Always use -strict. Without it, unknown fields are allowed, so a misspelled key is silently ignored. That is the same trap as in a real cluster, where a misspelled field can disappear without an error. kubeconform -strict turns on the strict schemas that reject unknown keys.
With a real cluster (a local kind or minikube cluster is enough), the same files can be applied, and then the commands in the next section show you what is going on.
Commands to know (reviewed, not run on a cluster here)
kubectl get service # list Services, see type and cluster IPkubectl get endpointslices # see the pod addresses behind each Servicekubectl describe service notes-api # selector, ports, and Endpoints linekubectl get ingress # list Ingresses and their addresskubectl describe ingress notes # rules, backends, and events from the controllerkubectl run tmp --rm -it --image=busybox:1.36 -- sh # a throwaway pod to test DNS from inside# inside it: nslookup notes-api and wget -qO- http://notes-apiThe last pair is the most useful test in the whole lesson. Most "networking problems" come down to one of: wrong selector (empty endpoints), wrong targetPort, a pod that is not ready, a DNS name typed in the wrong namespace, or a NetworkPolicy that blocks the call. Testing from inside the cluster with a throwaway pod separates "my Service is broken" from "my Ingress or laptop path is broken". Lesson 8 turns these checks into a full routine.
Try it
- A pod in namespace
shopwants to call a Service namedpaymentsin namespacebilling. Write two names that work. - A Service has
port: 80and notargetPort. The container listens on 3000. What happens, and what is the fix? - You created an Ingress and applied it. The cluster has no Ingress controller installed. What traffic is routed?
- Why can't a client reach a ClusterIP Service from a laptop outside the cluster, even if the laptop can reach the nodes?
- A StatefulSet needs each pod to be reachable by its own DNS name. Which Service type do you create?
- You have 3 pods behind a Service, and
kubectl describe serviceshows no endpoints. List three things to check. - Write an Ingress path rule so that
/shopand/shop/cartgo to Serviceshopbut/shoppingdoes not.
Answers
payments.billingandpayments.billing.svc.cluster.local.- The Service sends traffic to pod port 80 (it uses
portwhentargetPortis missing), where nothing listens. Connections are refused. Fix: settargetPort: 3000(or the named port). - None. An Ingress is only rules. You need a controller to act on them.
- The ClusterIP is virtual. It exists only as packet rewrite rules on the cluster's nodes, so it is not routable from outside. Use NodePort, LoadBalancer, Ingress, Gateway, or
kubectl port-forward. - A headless Service (
clusterIP: None), referenced by the StatefulSet'sserviceName. - (a) The Service selector matches the pod labels exactly. (b) The pods are Ready (readiness probe passing). (c) The Service and the pods are in the same namespace. Also check the pods exist at all.
path: /shopwithpathType: Prefix. Prefix matches by whole path elements, so/shopand/shop/cartmatch but/shoppingdoes not.
Interview questions
Why do we need a Service when pods already have IPs? (Must know)
Pod IPs are not stable. When a pod dies, its replacement gets a different IP, and a rollout or a scale-up changes the whole set. A Service gives clients one stable virtual IP and DNS name, and keeps the list of ready pod addresses behind it up to date by watching a label selector. The client talks to notes-api and does not care how many pods exist or where they run.
What are the Service types, and when would you use each? (Must know)
ClusterIP is the default and works only inside the cluster, so I use it for service-to-service calls. NodePort opens a port on every node, which I use for quick tests or as a base for something else. LoadBalancer asks the cloud for an external load balancer, which is good for exposing one thing but costs one load balancer per Service. ExternalName is only a DNS alias to an outside name. And a headless Service, with clusterIP: None, has no virtual IP, and DNS returns the pod addresses, which I use for StatefulSets and client-side balancing.
What is the difference between port, targetPort, nodePort and containerPort? (Must know)
port is where the Service listens, and what clients use. targetPort is the pod port the Service forwards to, and it can be a number or a name. nodePort is a port opened on every node for a NodePort Service, in the 30000 to 32767 range by default. containerPort is the port the app listens on and is mostly documentation. The classic bug is a missing targetPort: Kubernetes then reuses port, and traffic goes to the wrong pod port.
What is the difference between an Ingress and an Ingress controller? (Must know)
An Ingress is a Kubernetes object that holds routing rules: host and path to a Service. An Ingress controller is a program running in the cluster that watches those objects and configures a real proxy or load balancer to match. Without a controller, an Ingress does nothing at all. Also, controllers differ in the annotations they support, so an Ingress is only partly portable.
How does a pod find a Service by name? (Must know)
Through cluster DNS. Every Service gets a name like notes-api.default.svc.cluster.local. Each pod's resolver has a search list with its own namespace first, so the short name notes-api works inside the same namespace, and notes-api.other-namespace works across namespaces. There are also environment variables, but they are set only at pod start, so DNS is the better method.
A Service exists but requests fail, and the endpoints list is empty. What do you check? (Trap)
An empty endpoint list means no ready pod matches the Service. I check, in order, that the selector exactly matches the pod labels (a typo passes every schema check), that pods exist in the same namespace, and that they are Ready, because a failing readiness probe removes a pod from the endpoints. The trap is blaming the network or the Ingress first. If endpoints are empty, nothing else matters yet.
Is a Service a load balancer? How does it balance? (Senior signal)
Yes, but a simple one. For ClusterIP traffic each node's kube-proxy programs packet rules that rewrite the Service IP to one ready pod when a connection opens. There is no central proxy. It balances connections, not requests, and it does not look at load. So with long-lived connections like gRPC, one client may stay on one pod, and new pods get little traffic from old clients. For those I would use client-side balancing or a proxy that understands the protocol.
Ingress is "frozen". What does that mean, and what do you use for a new project? (Senior signal)
The Kubernetes docs say the Ingress API is generally available and will not be removed, but it gets no new development, and the project recommends the Gateway API. Gateway API is a set of add-on resources: GatewayClass, Gateway, HTTPRoute, GRPCRoute. It separates roles (platform team owns the Gateway, app teams own routes), supports things like header matching and traffic weights without controller-specific annotations, and is more portable. For an existing cluster full of Ingress objects I would keep them working and plan a gradual move. For a new design I would check whether my controller supports Gateway API.
What is a headless Service and why would you want one? (Senior signal)
It is a Service with clusterIP: None. It gets no virtual IP, kube-proxy ignores it, and DNS returns the addresses of the ready pods directly. I want one when clients need to know individual pods, for example members of a database cluster, or when the client does its own load balancing. StatefulSets need a headless Service to give each pod a stable DNS name.
Do NetworkPolicies protect me if I create them? (Trap)
Not necessarily. A NetworkPolicy is only an object. The network plugin (CNI) enforces it, and if the plugin does not support policies, nothing is blocked while everything looks configured. By default all pods can reach all pods. So I first confirm my CNI enforces policies, then I write a default-deny rule for sensitive pods and allow only the callers I expect. Then I test it by trying a connection that should fail.
A service must talk to something outside the cluster by a friendly name. What are your options? (Senior signal)
Simplest is a plain DNS name in config. If I want a stable in-cluster name that I can repoint later, I can make an ExternalName Service that returns a CNAME. A Service without a selector plus manual EndpointSlices also works for fixed IPs. I mention the limits of ExternalName: it is only DNS, so there is no port remapping, and with HTTPS the host name may not match the certificate.
When would you add a service mesh? (Senior signal)
When I have many services and a real need for something a mesh gives centrally: mutual TLS between all pods, uniform retries and timeouts, traffic splitting for canary releases, and per-call metrics. I would not add one for a few services. It costs memory and CPU for every proxy and adds operational complexity. First I would try timeouts and retries in code, NetworkPolicies, and a good Ingress or Gateway.
Key takeaways
- Pod IPs change. A Service is a stable name and virtual IP that finds pods by label selector.
- Only ready pods are in the EndpointSlice. Empty endpoints mean a selector typo or no ready pods.
- Default type ClusterIP works inside the cluster only. NodePort adds a port on each node. LoadBalancer asks the cloud for a load balancer. Headless (
clusterIP: None) returns pod IPs through DNS. - Inside the cluster, use DNS names:
service,service.namespace, or the fullservice.namespace.svc.cluster.local. - An Ingress is rules only, and needs an Ingress controller.
pathTypeis required. The Ingress API is frozen, and the Kubernetes project recommends the Gateway API for new work. - NetworkPolicy needs a CNI that enforces it. By default every pod can talk to every pod.
- A service mesh is for large systems with real needs. Do not start with one.
- A schema check passes a Service with a wrong selector. Use
kubeconform -strict, and then test in a real cluster.
What's next
Next: Configuration, Secrets and Storage in Kubernetes, Lesson 6. Your pods can now find each other. Next they need settings, passwords and disk space. You will meet ConfigMaps, Secrets (and why base64 is not encryption), and persistent volumes. Practise now: write the Deployment, Service and Ingress for one of your own apps, and run kubeconform -strict over them.
Published by Jainik · Last updated · How these lessons are made
Keep learning
Related lessons
- What Happens When You Type a URL (The Full Version) · The Browser Contract
- Containers, Kubernetes, Terraform and AWS for Full-Stack Interviews · Full-Stack Questions
Part of the study guide
Related series