Local Kubernetes Dev — Part 11: Networking — reaching your service and Ingress

By now you already have a built myapp image, manifests (a Deployment and a Service), and a running dev cluster on k3d. The service is most likely already running in the myapp namespace. But a natural question comes up: "How do I actually reach it?" Inside the cluster one Pod can see another, but from the browser on your laptop http://localhost:8080 leads nowhere — the cluster has its own network, and by default it's isolated from the host.

In this chapter we'll work through three layers of networking: how Pods find each other by name (DNS), the different ways to route traffic from the outside in, and how to set things up "like in prod" — through an Ingress with human-friendly domains such as myapp.localhost.

DNS inside the cluster: services by name

The first thing to understand: inside the cluster you reach services by name, not by IP. Kubernetes runs a built-in DNS server (by default this is CoreDNS) and automatically creates a DNS record for every Service. This is very convenient: a Pod's or service's IP can change when it's recreated, but the name stays stable.

The fully qualified (FQDN) name of an ordinary service follows this pattern:

1<service-name>.<namespace>.svc.<cluster-domain>

The default cluster domain is cluster.local. So our Service named myapp in the myapp namespace will be reachable at:

1myapp.myapp.svc.cluster.local

Yes, it looks redundant (the service name happens to match the namespace name), but that's fine. Such a name resolves to a ClusterIP — a stable virtual IP for the service, behind which traffic is load-balanced across Pods.

The good news: within a single namespace you don't need to write out the full FQDN. If PostgreSQL lives in the same myapp namespace under a service called postgres, then from myapp's code it's enough to specify the host postgres — the short name resolves thanks to the search domains configured in each Pod's /etc/resolv.conf. But from another namespace the short name won't work anymore — you'll have to write postgres.myapp or the full FQDN. This is a common trap: "but it worked for me," and then the service moved to a neighboring namespace.

The easiest way to test resolution is from inside a throwaway Pod:

1# Full name, works from any namespace
2kubectl run -it --rm dnstest --image=busybox --restart=Never -- \
3  nslookup myapp.myapp.svc.cluster.local
4
5# Short name — only from its own namespace
6kubectl exec -it -n myapp deploy/myapp -- nslookup postgres

A couple of details that will come in handy later (see the chapter on dependencies and the chapter on debugging):

The A/AAAA records of an ordinary (non-headless) service point to a single ClusterIP. For a headless service (clusterIP: None) the record returns the IPs of all Pods at once — which is a surprise for a client expecting a single VIP. Headless is usually needed for a StatefulSet, for example a database cluster where it matters to address specific instances.
For named ports, CoreDNS creates SRV records of the form _<port-name>._<protocol>.<service>.<namespace>.svc.cluster.local.

The details of the naming format are described in the official documentation, DNS for Services and Pods.

Ways to reach it from outside: port-forward, NodePort, Ingress

Now — how to route traffic from the host into the cluster. There are three main ways, and each has its own role.

port-forward — a tunnel for debugging

kubectl port-forward opens a temporary tunnel from your localhost to a single Pod (or to a Pod chosen behind a service). It's ideal for "quickly taking a look" or poking at an application you don't intend to publish.

1# Forward the service port (you can use the port name, e.g. https)
2kubectl port-forward -n myapp service/myapp 8080:8080
3
4# Or directly to a specific Pod
5kubectl port-forward -n myapp pod/myapp-xxxx 8888:8080

After this, curl http://localhost:8080/ goes to myapp. Don't confuse these two addresses: localhost:8080 is a direct port-forward tunnel to a Pod (that's how we opened the service in the chapter on Tilt and will poke at it in the chapter on debugging), while myapp.localhost:8081 below is the path through Ingress/Traefik. These are two different access mechanisms, not a contradiction. By default the tunnel binds only to loopback (127.0.0.1 and ::1); to open it for other machines on the network, add --address 0.0.0.0:

1kubectl port-forward -n myapp --address 0.0.0.0 deployment/myapp 8080:8080

Important limitations: the tunnel leads to a single Pod. When the Pod is recreated (rolling out a new version, scaling), the session drops — it's not load balancing and not "permanent" access, but a debugging tool. The syntax and behavior are described in the kubectl port-forward reference.

NodePort — a port on the node

A Service of type NodePort opens the same port on all cluster nodes in the range 30000–32767. Traffic on that port is forwarded into the service. The approach is simple, but the ports are nonstandard (no :80), and it doesn't look like prod at all.

Ingress — HTTP routing like in prod

An Ingress is a resource that describes HTTP(S) routing rules: which host and path lead to which service, and where TLS lives. This is how traffic usually flows in prod. But there's a critical nuance that almost every newcomer stumbles over. A quote from the documentation:

You must have an Ingress controller to satisfy an Ingress. Only creating an Ingress resource has no effect.

In other words, an Ingress manifest on its own is just a declaration of rules. For them to take effect, the cluster must have a running Ingress controller (Traefik, NGINX, etc.) that reads those rules and actually accepts traffic. Which controller serves a given Ingress is specified by the ingressClassName field.

Fortunately, in k3d the controller already comes out of the box — more on that below. Details about Ingress are in the official Ingress | Kubernetes. That page also mentions that Kubernetes is developing a newer Gateway API, but the classic Ingress is stable and frozen in its current form — for local development it's more than enough.

The Ingress controller in k3d (Traefik out of the box) and your own hosts

k3d is built on top of k3s, and k3s by default deploys Traefik as the Ingress controller on ports 80/443, plus a built-in ServiceLB (Klipper) — thanks to which LoadBalancer-type services don't hang forever in a pending status, as they would in a cluster without a cloud provider.

But there's one important step that's easy to skip. Traefik listens inside the cluster, and for it to be reachable from the host, you need to forward a port when creating the cluster to a special loadbalancer node (serverlb) that sits in front of the server nodes:

1k3d cluster create dev --api-port 6550 -p "8081:80@loadbalancer" --agents 2

Here -p "8081:80@loadbalancer" means: port 8081 on the host → port 80 of Traefik. Remember the key point: the port mapping is set only at cluster creation time. You can't add it to an already-running cluster — you'll have to recreate it. This is the second most popular source of pain after "forgot about the controller."

If you spun up the cluster in the k3d chapter without this flag, now is a convenient moment to recreate it with -p "8081:80@loadbalancer".

Now let's describe an Ingress for myapp. We assume you already have a Service named myapp on port 80 (which in turn leads to container port 8080):

1apiVersion: networking.k8s.io/v1
2kind: Ingress
3metadata:
4  name: myapp
5  namespace: myapp
6spec:
7  rules:
8  - host: myapp.localhost
9    http:
10      paths:
11      - path: /
12        pathType: Prefix
13        backend:
14          service:
15            name: myapp
16            port:
17              number: 80

Apply it and check:

1kubectl apply -f ingress.yaml
2curl http://myapp.localhost:8081/

The chain works out like this: curl → host port 8081 → Traefik (port 80 in the cluster) → by the rule host: myapp.localhost → Service myapp → Pod. This is exactly the routing scheme you'll have in prod, only instead of a real domain there's *.localhost, and instead of a cloud load balancer there's k3d. The recommended way to publish in k3d is described in Exposing Services - k3d.

I want my own Ingress controller

If for some reason you need not Traefik but, say, ingress-nginx, the built-in Traefik can be disabled at cluster creation and you can forward "bare" 80/443:

1k3d cluster create traefikless \
2  --k3s-arg "--disable=traefik@server:0" \
3  -p "80:80@loadbalancer" -p "443:443@loadbalancer"

For everyday local development this usually isn't needed — Traefik out of the box covers the job.

Local domains (*.localhost, editing /etc/hosts)

Where does it even come from that myapp.localhost points to your computer? It's not k3d magic. The .localhost TLD is reserved by the standard RFC 6761 for loopback: resolvers must return the loopback address (127.0.0.1 / ::1) for it and not send such queries to regular DNS.

In practice this means: Chrome and Firefox resolve any *.localhost (myapp.localhost, api.localhost, whatever) to loopback by themselves with no configuration at all — this is confirmed by community experiments too. Open http://myapp.localhost:8081 in Chrome and it just works.

But this is where the time-wasting begins. Not all clients can do this. Per Microsoft's documentation:

Safari on macOS doesn't support auto-resolution of *.localhost — it will go to the OS DNS stack and most likely find nothing.
Non-browser tools — curl, many CLIs, HTTP clients in code — treat *.localhost as an ordinary domain and also go to the operating system's DNS stack. If it doesn't resolve the name, you'll get could not resolve host.

So if you're testing through curl, scripts, or Safari (and also if you use custom domains not on .localhost), add an entry to /etc/hosts:

1echo '127.0.0.1  myapp.localhost' | sudo tee -a /etc/hosts

On Windows the file lives at C:\Windows\System32\drivers\etc\hosts (edited with administrator rights). After this, curl http://myapp.localhost:8081/ will work just like in Chrome.

So the rule is simple: in Chrome/Firefox *.localhost works right away; for curl, Safari, and any custom domains — add them to /etc/hosts.

When to choose which approach in local development

Let's put it all together. The three approaches aren't "better/worse" — they're different tools for different tasks.

Approach	When to use it	Downsides
port-forward	One-off debugging of a single service, quickly poking an API, accessing the DB from the host	Single Pod, drops on recreation, doesn't reproduce prod routing
Ingress (Traefik + *.localhost)	Everyday work, the main prod-like approach	Needs a controller and a port forward at cluster creation
NodePort	A fallback when you don't want to bother with Ingress	Nonstandard ports 30000–32767, least like prod

A practical recommendation for our `myapp`:

Main mode — Ingress. It reproduces exactly the traffic path you'll have in prod (host/path/TLS through a controller), gives you human-friendly domains, and works with multiple replicas. Combined with Tilt (see the chapter on Tilt), Ingress is configured once and then just lives on.
port-forward — for targeted debugging. Get into a specific Pod, check metrics, connect to PostgreSQL with a client from your laptop. Not for permanent access.
NodePort — if you really can't be bothered to set up Ingress in some one-off experiment. Mind the warning from the k3d documentation: forwarding a wide range of NodePort ports is resource-intensive and can hang the system because of how iptables rules are processed in Docker — forward only the specific ports you need.

The closer your local access is to production, the fewer surprises at rollout. So by default we choose Ingress and keep port-forward in our back pocket for debugging. How to inspect what's actually happening with traffic and Pods, we'll cover in the next chapter on debugging and observability.