XDP vs iptables: Dropping DDoS Before It Costs You

The first volumetric attack I ever fought, I fought with iptables and a text editor. This was before a CDN was something you could buy, when a hosting company’s border was its own problem: if something on the far side of the Internet decided to hate you, the mitigation plan was you, awake, blocking CIDRs by hand while the graphs climbed. I remember the particular despair of that loop. Block a range, watch the flood shift, block the next one, watch it shift again, and slowly understand that the rules were winning every argument and losing the war, because the box was executing my DROPs perfectly and going down anyway.

That night taught me the lesson this whole article sits on: the verdict is free, the arrival is not. A dropped packet has already crossed the wire, already taken its seat in the NIC’s ring, already burned an interrupt; the link, the router, and the firewall in front of the rule all paid full fare before the rule said no. REJECT is worse, because REJECT answers, and answering a flood is joining it. Nothing you write on the box can make traffic not arrive at the box.

Then I learned the trick that reaches upstream, and I can still feel how happy it made me: BGP. Announce the attacked address to your provider tagged with their blackhole community, and their border drops the flood at their edge, a hop before your building, at a scale where your attack is a rounding error. One route did what ten thousand rules could not, and we automated it within the week, because a script watching the counters announces that route faster than a human can wake up. The trade is brutal and worth saying plainly: a blackhole protects the body by finishing the attacker’s job on one address. Amputation, as a service. But it fixed the principle in me for good: place the refusal as early as you possibly can.

CDNs and scrubbing networks industrialized that upstream altitude; today you rent it by the month, and you should. The flood, meanwhile, has not gone anywhere. Booter services rent attacks by the hour, and the botnets behind them are assembled from hijacked cameras and DVRs, hardware nobody patches and nobody misses. And one class of it still lands on your own port and gets decided by your own kernel: the packet-rate flood, small frames at brutal frequency, comfortably inside your bandwidth and far outside your CPU. The question it asks is not whether your firewall can recognize junk. Any firewall can recognize junk. The question is what you pay per recognition, times fourteen million per second.

There, the altitude game replays in miniature. iptables and XDP can hold the exact same opinion about a hostile packet: drop it. The difference is the bill. By the time an iptables rule delivers that verdict, the kernel has already built the packet’s metadata object, taken it through connection tracking, and walked a rule chain to find the match; on a single core, that path saturates at one to two million packets per second. An XDP program delivers the identical verdict at the network driver, before the packet exists as anything the kernel will later have to clean up, and a single core drops roughly ten to twenty-four million per second. Under normal traffic nobody can tell these two firewalls apart. Under a packet-rate flood, that cost difference is the whole game.

The packet-rate variant found me years later wearing its best disguise: a healthy dashboard. An amplification flood was hammering an edge box, the bandwidth graph showed gigabits of headroom, every link light was green, and the server was unreachable: every core was pinned in softirq, and dmesg was scrolling nf_conntrack: table full, dropping packet faster than we could read it. The box was not out of bandwidth. It was out of verdicts.

The bill arrives before the verdict

To see why, follow one hostile packet into a Linux box and watch what the kernel spends on it before each mechanism gets its chance to say no.

The order is the whole story. The NIC writes the frame into a receive ring; that much is free, hardware does it. The first software to touch it is the driver, and that is where XDP lives: an eBPF program running against the raw buffer, before the kernel allocates the sk_buff, the metadata object every packet must wear to travel the stack. Drop here and the buffer is simply recycled back into the ring. Nothing was built, so nothing gets torn down.

Clear the driver and the meter starts. The sk_buff is allocated and initialized; tc’s ingress hook can drop the packet now, cheaper than what follows but already carrying the allocation. Keep descending and you enter netfilter, where iptables and nftables live: a connection-tracking lookup, or worse, a brand-new conntrack entry for every spoofed source address; then the hook chains, PREROUTING, INPUT, until your DROP rule finally matches. And a packet that nothing drops rides all the way up: socket lookup, queueing, a wake-up, a copy, so that recv() can hand your application sixty-four bytes of junk to ignore.

The DROP at the end refunds none of it. That is the property floods exploit.

What a drop costs, measured: iptables, nftables, tc, and XDP

In 2018 Cloudflare published a clean single-core experiment: a small-packet UDP flood steered at one receive queue, one CPU core, and each mechanism given the same job of making the traffic disappear. The ladder they measured is the diagram above made visible:

The academic measurement agrees and moves the ceiling. The XDP paper (Høiland-Jørgensen et al., CoNEXT 2018), on a 100 Gbps NIC, clocked 24 million drops per second per core for XDP against 4.8 million for the raw-table iptables path and 1.8 million with conntrack enabled, and the XDP number scaled linearly with cores to 115 million, where the PCI bus, not the software, became the wall. Divide the rates out and the per-packet arithmetic is stark: about 42 nanoseconds to say no in the driver, roughly half a microsecond to say the same word in netfilter’s conntrack path.

A fairness note before anyone reads this as a eulogy for netfilter, because nftables genuinely improved the world. One ruleset instead of four tools, atomic replacement, and typed sets and maps that turn what used to be ten thousand sequential rules into a single lookup: for real firewall policy, the kind with structure, it is a much better machine, and the people who maintain netfilter have carried an unglamorous load for two decades with care. Kudos, sincerely. But look at where its hooks sit. In Cloudflare’s bench, an nftables ingress drop clocked 1.53 million packets per second against 1.69 for the iptables raw table: a rounding error apart. The improvement is in expressing policy and scaling rulesets, not in the altitude of the hook. The sk_buff has already been paid for by the time nftables speaks, and no elegance in the ruleset gets that money back.

Stateful tracking adds a second failure mode under floods, nastier than throughput. Conntrack is a table, sized in memory, and a flood from randomized spoofed sources is a factory for entries that will never see a second packet. When the table fills, the kernel starts dropping whatever it cannot track, and it does not distinguish attack from customer: nf_conntrack: table full, dropping packet is the entire message you get. You can raise the limits and shorten the timeouts, and under a real flood you are tuning the rate at which you lose.

14,880,952

Ten-gigabit Ethernet, sixty-four-byte frames. Add the preamble and the enforced gap between frames and each one occupies 84 byte-times on the wire, 672 bits. Divide that into ten billion: 14,880,952 frames per second. Not a burst, not a record: the rated line speed of a port your fleet almost certainly exposes, sustainable indefinitely by hardware doing exactly what it was sold to do.

Now put the two halves of this article together. At the conntrack-path rate of 1.8 million drops per core, absorbing line rate costs better than eight fully consumed cores, every second, doing nothing but refusing. At the filter/INPUT rate from the Cloudflare bench it is more than twenty. At XDP’s measured rate, the same flood fits in roughly six tenths of one core. Same wire, same verdict; the difference is an entire server’s worth of CPU.

The attacker’s side of the ledger is insultingly cheap. Fourteen-point-nine million packets per second sounds industrial, but on the wire it is only ten gigabits: one rented machine with a decent NIC and a packet generator will produce it, which is exactly how the researchers produced it in the lab, or ten thousand hijacked cameras contributing one megabit each, about one video stream apiece, pointed at your address. DDoS headlines are denominated in terabits, and a ten-gigabit event does not make the news. It does not need to. Bandwidth kills links; packet rate kills CPUs.

Here is the human-scale version I keep in my head. Picture a corporate mail room during a junk-mail flood. The building can refuse sacks at the loading dock, one glance at the sack, straight into the dumpster, next; or it can accept everything, log each envelope in the register, sort it, load the cart, ride the elevator, and deliver each piece to a desk where the recipient glances at it and bins it. Both buildings threw away the same mail. One paid a glance per sack; the other paid the whole building. And the register is the quiet failure: it has finite lines, and once junk fills them, the clerk starts turning away real mail at the door. Netfilter is the desk. Conntrack is the register. XDP is the loading dock.

The CoNEXT authors ran the victim’s side of this experiment, and theirs are the numbers to remember. A server handling 35,000 TCP transactions per second was fed a rising small-packet flood. Filtering with the normal stack, its throughput was halved at 3 million packets per second of attack and effectively zero just below 3.5 million: less than a quarter of what a single 10GbE port delivers. The same server, with the same filter expressed as an XDP program, held roughly 28,500 transactions per second, steady, until the flood passed 19.5 million packets per second, beyond the port’s line rate entirely.

This is not a lab-only story. Cloudflare’s edge runs its volumetric mitigations exactly this way: L4Drop compiles attack signatures into eBPF and drops them in XDP, and during one production attack a single server shed more than eight million packets per second at a cost of roughly ten points of CPU. The mechanism you can deploy on a stock kernel this afternoon is the same one the biggest attack sponge on the Internet chose for the same job.

netfilter vs XDP, side by side

What XDP will not do for you

The honesty section, because a tool this sharp earns exaggerated claims, and exaggerated claims are how good tools get discredited:

• It is a scalpel for the packet-rate problem, not a firewall. XDP sees one packet at a time, early, with almost no context. It does not replace stateful filtering, NAT, or anything that reasons about flows, and it certainly does not replace an L7 WAF. Keep nftables for policy. Use XDP for the class of attack that kills netfilter: high-rate, low-value packets that deserve the cheapest possible no.
• XDP saves your cores, not your link. Every packet it drops still crossed the wire and filled the ring, which is the opening story’s lesson in modern clothes: if the attack is denominated in gigabits instead of packets per second, the port drowns before any on-host verdict is reached. That fight is won upstream, with the BGP blackhole and its industrial descendants, anycast CDNs and scrubbing networks. XDP owns the altitude between the wire and your stack, nothing above it.
• Early means blind. Everything netfilter gives you for free, connection state, rich match extensions, counters, logging, is absent at the driver hook. What you need, you rebuild in eBPF maps, and every feature you rebuild is code you now own, test, and debug with kernel tooling rather than a familiar rule syntax.
• The verifier is a strict landlord. eBPF programs must provably terminate and provably stay in bounds; complex parsing gets awkward, and some logic simply will not fit. This is the price of running untrusted-ish code in the driver path, and it is worth paying, but an iptables rule is one line while an XDP filter is software with a lifecycle.
• Driver support decides your real altitude. Native XDP needs explicit driver support; it is broad on server-class NICs today, but verify yours. The fallback, generic XDP, runs after the sk_buff is allocated, which quietly forfeits most of the reason you came. And at the top of the range, some SmartNICs can execute the program in NIC hardware itself, so the flood never even crosses the PCIe bus.
• The numbers here are bench numbers. Single cores, synthetic floods, minimum-size packets, 2018-era kernels and NICs. Your rates will differ. The ratio, an order of magnitude between netfilter’s altitude and the driver’s, has held up everywhere I have seen it measured, and it is the ratio that decides whether you survive.

Where this goes: eBPF at every altitude

Regular readers will recognize the family. In the workload identity series, a Cilium policy dropped an intruder’s SYN in the kernel because of who sent it, the last step of an identity that started at a database account. XDP is the same eBPF machinery pushed to the earliest point software can reach, the first instruction after the wire. One family, two jobs: policy where context is rich, survival where packets are cheap and hostile.

Strip away the mechanism and this article is one sentence, and it is the oldest sentence in capacity engineering: work you refuse early is capacity you keep. One blackhole route at the provider’s border, one XDP program at the driver: the same refusal, at different altitudes. A flood does not beat your firewall by outsmarting it. It beats your firewall by making it think fourteen million times a second, at half a microsecond a thought, until the thinking is the outage.

The cheapest packet to drop is the one you drop before you have paid to understand it.

A personal note. Somewhere between editing ifcfg files by hand and describing whole network topologies in netplan YAML, the old ceiling on Linux automation came down, and I have carried one invariant through every security and availability project of the last decade or more: defense in depth. Layers, always, because some layer is always having a bad day. What made me care about eBPF is not the speed. It is that the enforcement is out-of-band from the tenant’s point of view. In the old model, a network restriction inside the OS died with the first privilege escalation: root deletes the rule, the kernel-space implant outlives it, and the boundary was always a suggestion the attacker’s own kernel could withdraw. An XDP program on the host side of a tenant’s interface is a boundary the tenant cannot reach, no matter what it becomes inside its own kernel. I can hand a bad actor a root shell inside the guest and say it plainly: get out. Root buys everything inside the box, and the hook is not inside the box. No reach to the hook, no game.