Workload Identity on Kubernetes: Policy Without CIDRs

The rule that lets the application reach its database names an identity, not an address. No IP. No CIDR. No route. No security group. Written that way, the rule holds the same on one node or a thousand, on-prem or across three clouds, and it does not break when a pod moves, because nothing in it ever said where anything was.

In part 1 the same application connected to MySQL with no password, on one host. If part 1 was “the credential is not a string,” this is part 2: “the perimeter is not a subnet.” Same demo, same images, same page, same SQL. This time the application and the database live in different Kubernetes namespaces, and one new idea carries the whole piece: the network itself reads the identity.

The world we all grew up in: CIDRs, security groups, and routing gymnastics

Here is the system we are replacing, and again I will not caricature it, because I have run it for years. You want one true thing: the application may reach the database, and nothing else may. Now watch how many places you have to say it, and in what language.

Every one of those boxes is doing the same job: approximating “the app” with “an address the app happens to have right now.” That approximation is the bug. Addresses are a property of where a workload landed, not what it is, and in a modern fleet workloads land somewhere new constantly. The rules do not rot because anyone was careless. They rot because they encode a fact that was never stable.

Addresses rot at human scale too. Move to a new house and you spend a month restating the same fact to the bank, the DMV, the insurer, every subscription, each on its own form. Miss one and your mail keeps arriving at the old address, delivered faithfully to whoever lives there now. The postal system never asks who you are; it only knows where you were. An IP-keyed rule has exactly that failure mode, at fleet scale, with a database behind it instead of a mailbox.

Now make it worse, the way production actually makes it worse. Picture that fleet under Terraform, which is how any fleet this size is managed. One run decides a field forces replacement, or a data source re-reads to a new value, and a pod network comes back up in a different CIDR than it had yesterday. Nothing was destroyed on purpose. The plan looked routine. But every one of those six boxes still names the old range, and now half of them point at nothing while the other half point at whatever moved into that range next. On a single-region toy this is a five-minute fix. On a multi-region fleet at 2 a.m. it is a knot that pages everyone, and a terraform destroy fired at the wrong workspace turns the knot into a crater.

I have been in that room enough times to tell you what it actually sounds like, and it is never the technical part that hurts first. It is: do we call this a Sev 0 or not? What is the RTO on the thing that just went dark, and how much of the error budget did we spend getting here? What is this costing per minute, and is that a number we say out loud? Do we wake the VP now, or after we understand the blast radius? While that conversation runs, someone is on a host with root, and the fastest thing that makes the symptom stop is a chmod 777 and a permit-any rule, with a sticky note that says “fix this properly Monday.” The note never makes it back to the tracker. That is config drift at its worst: the distance between what your declarations swear is true and what a tired human typed at 3 a.m. to make the pager stop.

Identity does not make outages disappear. It removes this entire class of them. The rule named the workload, so when Terraform hands that pod a new CIDR, the rule is still correct, because it never mentioned the CIDR. Nothing to reconcile, nothing to drift, and nothing a cornered engineer is tempted to chmod 777, because the permission was never keyed to a number that just changed underneath them.

What changes when the network reads identity: SPIFFE, Cilium, and eBPF

The fix is the same move part 1 made at the application layer, pushed down into the datapath. In part 1 the database stopped asking “do you know the secret?” and started asking “what, verifiably, are you?” Here the network stops asking “what address are you coming from?” and starts asking the same question.

The machinery is already in place from part 1. SPIRE attests every workload and issues it a SPIFFE identity. On Kubernetes the attestation is k8s_psat: the SPIRE agent reads the pod’s projected ServiceAccount token and verifies it against the kubelet, so the identity is bound to the namespace and the service account: facts the control plane verifies, not claims the workload asserts about itself. Cilium, the CNI, gives every pod a compact numeric security identity derived from its labels, and enforces policy on that identity in eBPF, in the Linux kernel, on the packet path itself.

Put those together and the network can enforce a rule written in the language of identity. Not “traffic from 10.60.0.0/16 may reach 10.60.4.11.” Instead: “pods that ARE the webapp, in the app namespace, may reach the database.” When a pod reschedules to a new node and a new IP, nothing changes, because nothing in the rule was ever about the IP.

The demo design: one policy, zero addresses

Here is the entire cross-namespace access rule for the demo. Read it the way I did the first time, looking for the part that will need editing when the fleet grows. There isn’t one.

Those two annotations are where a platform team’s on-call load changes. The rule names who, and “who” is a stable fact that travels with the workload. There is no range to widen when you add capacity, no rule to re-key when a pod moves, no per-region copy drifting out of sync. The policy you wrote on day one is the policy that holds at a thousand nodes. That is not a scaling optimization. It is the difference between a rule that describes your intent and a rule that describes last Tuesday’s IP assignments.

And it composes with part 1 rather than replacing it. The database still demands a valid client certificate. So the demo now has two independent gates, each keyed to identity, at two different layers.

The intruder in the demo is the sharpest way to see it. It runs the identical image and code as the webapp, in a different namespace. Before the policy, it can open a raw TCP socket to the database’s port; only part 1’s certificate gate stops it there. After the policy, its very first SYN packet is dropped by eBPF before a single byte of TLS is exchanged. Same bytes, same cluster, wrong identity, refused by the network itself.

The build: the same passwordless app across Kubernetes namespaces

Standing this up was the short part, and that is worth a sentence, because it is a property of the series, not luck. The lab from part 1 was still warm: SPIRE server and agents attesting, the passwordless MySQL, the webapp image, all of it already proven. This piece added three namespaces and one policy. The network isolation knobs were the only new work, and proving them was a short run, not a rebuild.

I reused the part 1 payloads without editing a single line, and it is worth being precise about why, because read quickly it can look like a trick, and it is not one. Kubernetes gives every Service a structured DNS name, service.namespace, addressable at any depth: mysql, mysql.wi-demo, and mysql.wi-demo.svc.cluster.local all resolve to the same place. Those dotted levels are human organization, a folder path for services, suffixes you append or drop to say how specific you are being, and you can nest them as deep as your naming scheme wants. I named the database Service mysql inside namespace wi-demo, so its in-cluster name is mysql.wi-demo, which is exactly the hostname part 1’s page, its certificate SANs, and its CREATE USER already used. The payloads ran unchanged because the name lined up. That is the whole mechanism, and it is naming, not magic.

Say plainly what is not happening, because this is where a careful reader should get suspicious: the DNS name is not a secret and it does no security work. Every pod in the cluster can resolve mysql.wi-demo; the intruder did. Resolving it buys nothing. The certificate subject and the network policy both name the workload’s identity, and neither trusts the resolver. Spoof the name, guess it, own the cluster DNS server outright, and you still hold no SVID and the policy still drops your packet. The name is an address, and this whole series has exactly one rule about addresses: they are for finding things, never for trusting them.

Three namespaces set the scene: wi-demo (the database), wi-app (the application), and wi-intruder (the same image and code, in the wrong place). The identity policy is two SPIRE registration entries. Where part 1 gated on the machine’s TPM and the exact image digest, the Kubernetes-native gate is the namespace and the service account, facts the SPIRE agent verifies against the kubelet:

spire-server entry create -parentID .../node/cluster-a \
  -spiffeID .../wi/webapp \
  -selector k8s:ns:wi-app -selector k8s:sa:webapp \
  -dns webapp.wi-demo -x509SVIDTTL 900

The webapp comes up in wi-app, fetches its own 15-minute SVID over the Workload API socket the SPIRE agent DaemonSet exposes on every node, and connects across the namespace boundary to mysql.wi-demo. The database certifies it exactly as before, with an account that has no password, only a required certificate subject. The page is the same page from part 1, now reporting its Kubernetes gates:

Nothing on that page is new except the flavor of the gates: node attested via k8s_psat, k8s ns wi-app + sa webapp. The identity issuance, the mutual TLS, the empty-password account, the fifteen-minute lifetime, all carried over from part 1 untouched. That is the point I keep coming back to: the workload never learned it was on Kubernetes. It asks the local socket “who am I?” and gets an answer. Where it runs is somebody else’s problem.

What the network refuses: the intruder, dropped in the kernel

A network-policy demo that only shows the allowed path is a toy, same as before. Here is the refusal, and it is the best artifact in the piece because it is packet-level, not a claim about YAML.

Before the policy, the flat-network reality every Kubernetes operator feels in their gut: the intruder opens a TCP connection straight to the database, in a different namespace, unbothered. Kubernetes is flat by default. Then I applied the one policy above and ran the same probe. Cilium’s flow log, verbatim:

wi-intruder/intruder-…:41960  <>  wi-demo/mysql-…:3306 (ID:84360)  policy-verdict:none  DENIED (TCP Flags: SYN)
wi-intruder/intruder-…:41960  <>  wi-demo/mysql-…:3306 (ID:84360)  Policy denied  DROPPED (TCP Flags: SYN)

Read the identity numbers, not the addresses. ID:84360 is the database’s Cilium security identity; the drop is decided by comparing the sender’s identity against the policy, in eBPF, on the SYN. The connection is refused before the TCP handshake completes, long before TLS, and the probe simply times out. The webapp, whose identity the policy names, stays connected throughout. Two independent gates, and the intruder does not clear the first one.

Since we are already inside the kernel: the same eBPF machinery, moved down to the network driver hook, will also refuse a volumetric flood at line rate for a fraction of a core. Different job, same tools, and it deserves a measured treatment of its own, so I will put XDP against iptables in a coming piece and count the cost per verdict, because that is the cleanest way to see how efficient the control you just watched really is, how early in the stack it can act, and how that earliness becomes resilience for the stack and availability for the services behind it when a volumetric flood arrives. One number frames the stakes: more than 70 percent of today’s Internet traffic is automated, machines talking to machines, bots, crawlers, and replayed scripts, not a human on the other end. When most of what knocks is not a person, where you refuse work stops being a tuning detail and becomes the capacity plan.

In the old world that same intruder pod would have worked. It would have had a routable address inside the cluster’s flat network, reached 3306, and, if it had scraped the database password from anywhere, logged in. The network had no way to tell it apart from the real app, because the network was reasoning about addresses, and the intruder had a perfectly valid one.

That is not an edge case. It is the shape of nearly every serious intrusion. The breach almost never starts at the crown jewels; it starts at something small and reachable, a stale web pod, a leaked token, a phished laptop, and becomes a breach only when the attacker moves from there to something that matters. That hop has a name, lateral movement, and it is present in the large majority of real-world breaches. The industry’s own numbers sharpen the point: CrowdStrike’s recent threat reporting finds that the majority of detections are now malware-free, meaning the attacker is not dropping a virus, they are logging in with valid credentials and using the tools already on the box, which is to say they look exactly like the intruder in this demo, a legitimate image with a legitimate address. And breakout time, the gap between first foothold and first lateral move, has collapsed to minutes, in the fastest observed cases seconds. Nobody is fast enough to catch that by hand. Containment has to be structural, or it does not exist.

Now weigh what it costs to stop the hop in each world. Stopping lateral movement with addresses means microsegmentation, and microsegmentation drawn in addresses is a wall built one brick at a time: a rule per allowed flow, hundreds of them in a real fleet, each an IP pair that goes stale the instant a pod reschedules, which is the 3 a.m. war room from earlier on a loop. Most teams never finish that wall, and the half-built version is worse than none, because it reads as done. Here it is one line. The policy names who may reach the database and denies everyone else by default, so the lateral hop from a compromised namespace does not fail because we enumerated it as forbidden; it fails because we never authorized it. The stolen password and the perfectly valid address are both worthless, because neither is the thing being checked. One small declaration closes a class of attack that a few hundred hand-kept firewall rules, in a less-than-ideal architecture, would not reliably close at all.

CIDR firewalls vs identity policy, side by side

What the demo simplifies (and what production would add)

An introduction that hides its simplifications teaches the wrong thing, so, plainly:

• The network gate is real and packet-proven. The DROPPED verdict above is Cilium’s own flow log with the security-identity numbers, not policy YAML that merely claims to deny. The kernel dropped the SYN.
• k8s_psat binds identity to namespace and service account, verified against the kubelet. That is the Kubernetes-native equivalent of part 1’s TPM-plus-image gate, and it is honest about what it proves: this pod runs under this service account in this namespace. The container image is also available as a selector (k8s:container-image, the direct analog of part 1’s digest pin); the demo gates on namespace and service account to keep the namespace story in front.
• Cilium can also do mutual authentication of its own, layering Cilium-managed SPIFFE mTLS onto the network path so the identity is cryptographically proven at connection time, not only inferred from labels. I did not enable it here. The lesson of part 2 is the identity-named policy; adding Cilium’s mTLS on top is the next turn of the screw, and it is a deliberate stretch goal rather than a dependency, because it upgrades a substrate I wanted to leave proven.
• This ran on a single cluster. The rule is written to be indifferent to that, and Cilium ClusterMesh extends the same identity across clusters and regions, but I have not shown two clusters wired together in this demo. The claim I am making is scoped to what the flow log proves.

Where this goes: multi-cluster identity and the region-agnostic fleet

Strip away the tooling and part 2 is one sentence: stop authorizing where a workload is, start authorizing what it is, all the way down to the packet. The perimeter stops being a subnet you draw and becomes a name that travels with the workload.

The reason this matters more the bigger you get is the reason the old diagram had a “new region” pain box. Address-based segmentation has a cost that scales with your footprint: every cloud, every region, every cluster is another place to restate the same intent in another dialect, and another place for it to drift. Identity-based policy has a cost that scales with the number of distinct intents you actually have, which is small and stable. You wrote “the app may reach the database” once. It does not matter how many regions you run it in.

That is the property I was chasing across both articles. Part 1 removed the password. Part 2 removed the address. What is left, in both cases, is a name that a workload earns by proving what it is, and that the whole stack, from the database account to the kernel datapath, agrees to honor for exactly as long as you say.

The database still does not know any passwords. Now the network does not know any addresses either.