Local Kubernetes Dev — Part 6: Containerizing your service — writing a Dockerfile

Anatomy of a Dockerfile, using myapp (FastAPI) as the example

A Dockerfile is a sequence of instructions, each describing one build step. The most common ones:

• FROM — the base image we build on top of (for example, a ready-made image with Python).
• WORKDIR — the working directory inside the image.
• COPY — copying files from your machine into the image.
• RUN — running a command during the build (installing packages, etc.).
• ENV — environment variables.
• EXPOSE — documenting the port the application listens on.
• USER — which user the process runs as.
• CMD — the command that runs when the container starts.

Let's start with a simple, "naive" version — it works, but it's not ideal. Suppose the project structure looks like this: the code lives in an app/ directory (with app/main.py containing the FastAPI object inside), and dependencies are listed in requirements.txt.

So the example is reproducible from the very start, here's a minimal myapp. The app/main.py itself (we'll cover the full version of the probes with a real PostgreSQL check in chapter 13):

1# app/main.py
2from fastapi import FastAPI
3
4app = FastAPI()
5
6
7@app.get("/")
8def root():
9    return {"service": "myapp"}
10
11
12@app.get("/healthz")   # liveness: the process is alive
13def healthz():
14    return {"status": "ok"}
15
16
17@app.get("/ready")     # readiness: ready to accept traffic (we'll add a DB check in chapter 13)
18def ready():
19    return {"status": "ready"}

1# requirements.txt
2fastapi
3uvicorn[standard]

Now the Dockerfile. Note: WORKDIR /code + COPY ./app ./app place the code in /code/app — we'll need this path again in the Tiltfile (chapter 8).

1FROM python:3.12-slim
2
3WORKDIR /code
4
5# Useful environment variables for Python in a container
6ENV PYTHONDONTWRITEBYTECODE=1 \
7    PYTHONUNBUFFERED=1 \
8    PIP_NO_CACHE_DIR=1
9
10COPY requirements.txt .
11RUN pip install --no-cache-dir -r requirements.txt
12
13COPY ./app ./app
14
15EXPOSE 8080
16
17CMD ["fastapi", "run", "app/main.py", "--port", "8080"]

Let's go through the key points.

fastapi run instead of bare uvicorn. The official FastAPI containers guide recommends exactly the fastapi run command — under the hood it starts the same Uvicorn, but with sensible production settings (FastAPI in Containers). Locally, for hot reload, you run uvicorn --reload, but in the cluster image you must not use --reload: it's extra overhead and a potential source of leaks, and the mode is intended for development only. (We'll set up the fast "edit code → restart" loop in the cluster differently — with Tilt, see chapter 8.)

The exec form of CMD. Notice that the command is written as an array of strings (["fastapi", "run", ...]), not as a single string. This is critical: with the exec form, the application process becomes PID 1 and receives system signals directly. If you write the shell form (CMD fastapi run app/main.py), the command gets wrapped by /bin/sh, the SIGTERM signal from Kubernetes won't reach the application, and graceful shutdown (and with it FastAPI's lifespan hooks) will break — the pod will be killed hard on timeout.

Environment variables. PYTHONDONTWRITEBYTECODE=1 disables writing .pyc files, PYTHONUNBUFFERED=1 forces logs to go straight to stdout (otherwise they get stuck in the buffer and you won't see them in kubectl logs), and PIP_NO_CACHE_DIR=1 keeps pip from bloating the image with its cache.

If your service will run behind a TLS proxy (and in the cluster it almost certainly will — via Ingress, see chapter 11), add the --proxy-headers flag so FastAPI correctly reads X-Forwarded-*:

1CMD ["fastapi", "run", "app/main.py", "--proxy-headers", "--port", "8080"]

About the number of workers: the classic combination is Gunicorn as the process manager and Uvicorn as the workers, with the worker count usually estimated by the heuristic (2 * CPU) + 1. But in Kubernetes it's often simpler to run one worker per pod and scale with replicas — that way the cluster itself manages the load. For local development this isn't even a question: a single process is enough.

Multi-stage builds: separating build from runtime

The naive image above drags everything into the final result: compilers, dev headers, pip caches. For runtime that's dead weight. A multi-stage build solves the problem: in a single Dockerfile you can write several FROM sections, each one a separate build stage, and copy only the finished artifacts into the final image (Multi-stage builds — Docker Docs).

Stages can be named (FROM ... AS build), and you copy between them with COPY --from:

1# --- Stage 1: build dependencies ---
2FROM python:3.12-slim AS build
3
4ENV PYTHONDONTWRITEBYTECODE=1 \
5    PYTHONUNBUFFERED=1 \
6    PIP_NO_CACHE_DIR=1
7
8WORKDIR /code
9
10# Install dependencies into an isolated venv so we can move it as a whole
11RUN python -m venv /opt/venv
12ENV PATH="/opt/venv/bin:$PATH"
13
14COPY requirements.txt .
15RUN pip install --no-cache-dir -r requirements.txt
16
17# --- Stage 2: runtime ---
18FROM python:3.12-slim
19
20ENV PYTHONDONTWRITEBYTECODE=1 \
21    PYTHONUNBUFFERED=1 \
22    PATH="/opt/venv/bin:$PATH"
23
24WORKDIR /code
25
26# Copy ONLY the finished venv from the build stage — no pip caches or compilers
27COPY --from=build /opt/venv /opt/venv
28COPY ./app ./app
29
30EXPOSE 8080
31
32CMD ["fastapi", "run", "app/main.py", "--port", "8080"]

The idea is simple: everything "dirty" (installing packages, possible compilation) happens in the build stage, while only the /opt/venv directory with the already-built libraries moves into the runtime image. The Docker documentation notes that COPY --from can pull files not only from your own stages but also from external images (for example, COPY --from=nginx:latest ...).

How much this shrinks the image depends on how many build dependencies you have. If packages install from ready-made wheel files, the gain is modest; if something compiles from source, the savings can be substantial (on the order of tens of percent and more). We can't promise an exact number, but the direction is always the same: the final image becomes smaller and cleaner.

A handy debugging trick: stop the build at a specific stage and look inside.

1docker build --target build -t myapp:builder .

(myapp:builder here is a temporary debugging tag, to avoid confusing the intermediate stage with our main myapp:dev image.)

Layer caching: why instruction order matters

Each Dockerfile instruction creates a layer. Docker caches layers and, on a rebuild, reuses the ones that haven't changed. The key rule: if some layer changes, all the layers after it are invalidated too — as the Docker documentation puts it, "if a layer changes, all the layers that follow it are affected" (Docker build cache).

From this follows the main principle: put what rarely changes earlier, and what changes often later. Most often you edit the application code, not the list of dependencies. So dependencies should be installed before copying the code. That's exactly what we did above:

1COPY requirements.txt .
2RUN pip install --no-cache-dir -r requirements.txt   # heavy layer, rarely changes
3COPY ./app ./app                                      # light layer, changes often

The FastAPI guide describes this trick directly: "first copy the file with the dependencies separately, not the whole code at once." Installing dependencies "can take minutes," but from the cache it's "seconds at worst." If you do the opposite — put COPY . . before installing dependencies — then any one-line edit in the code will invalidate the code layer, and after it the pip install layer, and Docker will reinstall all libraries from scratch on every build. This is the most common anti-pattern.

One more caching rule concerns system packages: always combine the apt-get update and apt-get install commands into a single RUN. Otherwise a cached, stale apt-get update will lead to installing outdated versions:

1RUN apt-get update && apt-get install -y --no-install-recommends \
2        libpq5 \
3    && rm -rf /var/lib/apt/lists/*

Lightweight base images (slim/alpine/distroless) and their pitfalls

The choice of base image affects size, compatibility, and security. Three popular options for Python:

slim (debian-slim). A trimmed-down Debian with glibc and the apt package manager. Best compatibility: the vast majority of Python packages have ready-made binary wheel files for glibc, so nothing gets compiled. This is a reasonable default choice — it's exactly what we used (python:3.12-slim).

alpine. A very small image (Alpine itself is on the order of a few megabytes, Docker best practices). But there's a catch: Alpine uses musl libc instead of glibc. There are no standard manylinux wheel files for musl, so pip often compiles packages from source — that's slow and fragile, especially for scientific libraries. Plus the historical quirks of how DNS works in musl. The conclusion: use Alpine for Python only if you're sure all your dependencies get along with it; in other cases slim is more practical.

distroless. Images from Google that contain "only your application and its runtime dependencies" and deliberately do not contain "package managers, shells" (GoogleContainerTools/distroless). Fewer packages means fewer potential vulnerabilities (CVEs). This is a great choice for a secure production runtime, but almost always paired with multi-stage: you build dependencies on a regular slim and copy them into distroless.

1# Build stage — as before, on python:3.12-slim
2FROM python:3.12-slim AS build
3# ... install dependencies into /opt/venv ...
4
5# Runtime on distroless
6FROM gcr.io/distroless/python3-debian12
7WORKDIR /code
8COPY --from=build /opt/venv /opt/venv
9COPY ./app ./app
10ENV PATH="/opt/venv/bin:$PATH"
11EXPOSE 8080
12# There's no shell in distroless — exec form only!
13ENTRYPOINT ["python", "-m", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]

The pitfall of distroless: there's no shell inside, so docker exec -it ... sh won't work, and CMD/ENTRYPOINT must be in exec form. For debugging, use the :debug variants of the image (they include a busybox shell) or kubectl debug (see chapter 12). For your first encounter and for local development I recommend staying on slim — it's easier to debug. It makes sense to switch to distroless later, when preparing the image for production (see chapter 13).

.dockerignore, an unprivileged user, EXPOSE

.dockerignore. When you run docker build, Docker first sends the entire "build context" — the contents of the current directory — to the builder. The .dockerignore file excludes the unnecessary parts from it; it works by the same rules as .gitignore (Docker best practices). Without it you risk bloating the context and, worse, accidentally pulling .git, a local virtual environment, or secrets like .env into the image.

1.git
2.gitignore
3__pycache__/
4*.pyc
5.venv/
6venv/
7.env
8.pytest_cache/
9.mypy_cache/
10*.md
11Dockerfile
12.dockerignore

An unprivileged user (USER). By default a process in a container runs as root — this violates the principle of least privilege. The Docker documentation advises: "if a service can run without privileges, use USER." We create a regular user with an explicit UID and switch to it before startup:

1RUN adduser --disabled-password --uid 10001 appuser
2USER appuser

In Kubernetes this is later complemented by setting securityContext.runAsNonRoot: true in the pod manifest (see chapter 7 and chapter 13) — the image and the cluster reinforce each other.

EXPOSE. It's important to understand: EXPOSE 8080 does not publish the port to the outside. It's only metadata — documentation about which port the application listens on (Docker best practices). The actual publishing is done with the -p flag in docker run or with Service/Ingress objects in Kubernetes (see chapter 11). The FastAPI guide gets by without EXPOSE at all, but it's useful to keep it as a hint to whoever reads the Dockerfile.

A good idea is to add a HEALTHCHECK so Docker knows whether the service is alive (in Kubernetes there are separate probes for this, but locally HEALTHCHECK is handy). An important detail: python:3.12-slim (and even more so distroless) has no curl, so we do the check with Python itself, which is definitely present in the image:

1HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
2  CMD ["python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8080/healthz').getcode()==200 else 1)"]

Testing the image locally and loading it into the k3d registry

First we build and test the image locally, with plain Docker:

1docker build -t myapp:dev .
2docker run --rm -p 8000:8080 myapp:dev

Now the service is available at http://localhost:8000 (port 8000 on the host is mapped to 8080 inside the container). It's also useful to look at the size and layers:

1docker images myapp:dev
2docker history myapp:dev

Path 1: direct import (k3d image import). The fastest way for a one-off test is to push an already-built image straight into the cluster nodes (Importing images — k3d):

1docker build -t myapp:dev .
2k3d image import myapp:dev -c dev

You can import several images at once or load from a tar archive:

1k3d image import myapp:dev myworker:dev -c dev
2docker save myapp:dev -o myapp.tar && k3d image import myapp.tar -c dev

In the manifest you must then set imagePullPolicy: IfNotPresent — otherwise Kubernetes will still try to pull the image from the network and won't find it:

1image: myapp:dev
2imagePullPolicy: IfNotPresent

Path 2: the built-in registry. For ongoing work a local registry is more convenient — a private "image warehouse" inside k3d. Then the cycle is the familiar one: build → push → the cluster pulls the image itself (Using Image Registries — k3d). We already brought up our canonical registry k3d-registry.localhost:5000 together with the dev cluster in chapter 5 — there's nothing to recreate.

Now we build, tag for the registry address, and push from the host to k3d-registry.localhost:5000:

1docker build -t k3d-registry.localhost:5000/myapp:dev .
2docker push k3d-registry.localhost:5000/myapp:dev

The convenience of the built-in k3d registry is that the same name k3d-registry.localhost:5000 works both from the host (for push) and from inside the cluster (for pull). That's why in the chapter 7 manifest the image is specified with exactly the same address:

1image: k3d-registry.localhost:5000/myapp:dev

If you created the registry without an explicit port (--registry-create k3d-registry.localhost), k3d will assign a random port — you can find it out with docker ps:

1docker ps -f name=k3d-registry.localhost

In practice you'll hardly have to do any of this by hand: in the next chapters Tilt will take over building, pushing, and updating the pod (see chapter 8). But it's important to understand what's happening under the hood — when something goes wrong, you'll know where to look.

So, we have a tidy, small, and secure myapp image, and it's sitting in the cluster. Next, we'll teach Kubernetes to run it: on to the manifests.

Sources

• FastAPI in Containers - Docker (official guide)
• Multi-stage builds — Docker Docs
• Docker build cache — Docker Docs
• Building best practices — Docker Docs
• GoogleContainerTools/distroless (official repository)
• Using Image Registries — k3d (official documentation)
• Importing images — k3d (official documentation)
• How to Use Docker Images with k3d (k3s in Docker) — oneuptime