Hardening the API server & kubelet

anonymous-auth, authorization-mode, kubelet webhook.

Advanced14 min · lesson 6 of 24

The API server is the single front door to the cluster: every kubectl command, every controller, every kubelet, and every attacker goes through it, and it is the one component that reads and writes etcd. Every request runs the same pipeline — authenticate (who are you), authorize (are you allowed), admission (should this specific write be mutated or rejected), then persist. A handful of flags on that server decide how much of this is actually enforced, and getting them wrong exposes the whole cluster at once.

THE API SERVER REQUEST PIPELINE
1AuthenticateWho are you? anonymous-au…2AuthorizeAre you allowed?…3AdmissionMutate or reject this…4PersistWrite to etcd
Every kubectl call, controller, kubelet, and attacker runs this same pipeline; a few API-server flags decide how much of it is actually enforced.

The non-negotiables: turn off anonymous auth so unauthenticated requests are rejected rather than mapped to system:anonymous; set authorization to Node,RBAC so the real authorizers run (never AlwaysAllow); enable audit logging so there is a record; and switch off profiling, which exposes internals on the debug endpoints. Modern versions already removed the old insecure port, but confirm it. These live in the static pod manifest, so saving the file restarts the API server within seconds.

/etc/kubernetes/manifests/kube-apiserver.yaml
spec:
containers:
- command:
- kube-apiserver
- --anonymous-auth=false
- --authorization-mode=Node,RBAC
- --profiling=false
- --service-account-lookup=true # honor token revocation
- --audit-log-path=/var/log/kubernetes/audit.log
- --audit-log-maxage=30
- --tls-min-version=VersionTLS12
One bad flag opens everything
--authorization-mode=AlwaysAllow disables authorization entirely — anyone who reaches port 6443 can do anything. --anonymous-auth=true maps unauthenticated calls to a real (if limited) identity. These two are the highest-impact misconfigurations kube-bench flags, and they are exactly what an attacker probes for first.

The kubelet is the soft underbelly

Each node runs a kubelet with its own HTTPS API on port 10250, and its historical defaults are the weakest link in a cluster: anonymous access allowed and authorization set to AlwaysAllow means anyone who can reach the port can list pods and exec into containers on that node. Lock it down in the kubelet config file: disable anonymous auth, require the API server’s CA for client certs, switch authorization to Webhook so each call is checked against the API server, and set the read-only port to 0.

/var/lib/kubelet/config.yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
anonymous:
enabled: false
webhook:
enabled: true # bearer tokens checked via the API server
x509:
clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
mode: Webhook # not AlwaysAllow
readOnlyPort: 0 # close the unauthenticated :10255 port

Restart the kubelet to apply (systemctl restart kubelet), then prove the lockdown from the node: an unauthenticated call to the kubelet API should now be refused instead of dumping the pod list. That one curl is the difference between “I set the flag” and “the flag works.”

terminal
$ sudo systemctl restart kubelet
$ curl -sk https://localhost:10250/pods | head -c 40
Unauthorized # before hardening this returned the full pod list
A typo here crashes the control plane
The API server manifest is live: save an unknown flag or a bad value and the container crash-loops, and kubectl stops answering because the thing serving kubectl is down. Keep a backup copy outside /etc/kubernetes/manifests (a live copy there would start a second API server), and if it fails to come up, read the container log with crictl logs.