Container immutability at runtime

Read-only rootfs, dropped caps, no shell.

Advanced10 min · lesson 23 of 24

An immutable container never changes after it starts: no new binaries dropped in, no config rewritten, no writable root filesystem. That is both a hardening posture and a detection signal at once. As hardening, it removes the attacker’s ability to install tools or tamper with the app in place; as detection, it turns any write to disk or any spawned shell into a high-confidence alarm, because a correctly immutable container should never do those things.

You make a container immutable with the same security-context fields as before, anchored by readOnlyRootFilesystem: true. The catch is that almost every real app needs to write somewhere — a temp dir, a cache — so you give it explicit, named writable mounts (emptyDir or a volume) instead of a writable root. That makes every writable location visible in the spec and auditable, rather than “anywhere.”

pod.yaml
spec:
containers:
- name: app
image: registry.internal/app@sha256:9f2a... # pinned by digest
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
runAsNonRoot: true
capabilities: { drop: ["ALL"] }
volumeMounts:
- { name: tmp, mountPath: /tmp }
- { name: cache, mountPath: /var/cache/app }
volumes:
- { name: tmp, emptyDir: {} }
- { name: cache, emptyDir: {} }

Enforce it, then watch it

Require the immutable settings so nobody ships a writable, over-capable pod by accident — with Pod Security’s restricted profile for the baseline, or a Kyverno/Gatekeeper policy that specifically mandates readOnlyRootFilesystem. Then let Falco catch any violation at runtime: a write below a binary directory, or a shell in a distroless image, becomes a precise alert precisely because the container is supposed to be inert.

kyverno-readonly.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: { name: require-ro-rootfs }
spec:
validationFailureAction: Enforce
rules:
- name: readonly-root
match: { any: [{ resources: { kinds: [Pod] } }] }
validate:
message: "readOnlyRootFilesystem must be true"
pattern:
spec:
containers:
- securityContext: { readOnlyRootFilesystem: true }
An app that writes needs a named mount
readOnlyRootFilesystem breaks any app that writes to its own root, and the failure is often a cryptic crash at startup. The fix is not to abandon it — it is to find the paths the app writes (run it once, watch the errors) and grant each an explicit emptyDir or volume, so every writable location is declared and nothing else is.