I ran out of AI. Not electricity. Not water. Not internet. AI. I had exhausted the allowance on one of the highest plans available from a major lab, and the reset was still hours away. I pay for top plans across the major providers, so another endpoint was available. I also spend money every month on end-user APIs for small projects that do nothing glamorous. They remove friction from my life, one repetitive task at a time. ...
The first volumetric attack I ever fought, I fought with iptables and a text editor. This was before a CDN was something you could buy, when a hosting company’s border was its own problem: if something on the far side of the Internet decided to hate you, the mitigation plan was you, awake, blocking CIDRs by hand while the graphs climbed. I remember the particular despair of that loop. Block a range, watch the flood shift, block the next one, watch it shift again, and slowly understand that the rules were winning every argument and losing the war, because the box was executing my DROPs perfectly and going down anyway. ...
The rule that lets the application reach its database names an identity, not an address. No IP. No CIDR. No route. No security group. Written that way, the rule holds the same on one node or a thousand, on-prem or across three clouds, and it does not break when a pod moves, because nothing in it ever said where anything was. In part 1 the same application connected to MySQL with no password, on one host. If part 1 was “the credential is not a string,” this is part 2: “the perimeter is not a subnet.” Same demo, same images, same page, same SQL. This time the application and the database live in different Kubernetes namespaces, and one new idea carries the whole piece: the network itself reads the identity. ...
The application connects to MySQL without a password. Not “the password is in a vault.” Not “the password is injected at deploy time.” Not “no password needed because the request comes from an allow-listed IP.” No password at all. There is no string, anywhere, that logs into that database. That sentence is where I recently lost a friend, a senior engineer at one of the biggest tech companies you can name, while explaining my fleet-security work. He made me say it three times. And he is not behind; he is normal. Zero-trust workload identity is still a novelty to most people who run real infrastructure, including very good ones. ...
A friend called last week with a familiar complaint. He had built his workload inside AWS Nitro Enclaves, and he wanted out. His words, not mine: “Pretty easy to get in. Pretty costly to get up. Impossible to get out.” A friendly onboarding pipeline had generated his key for him and left it sitting right there in the console, and he honestly could not tell you whether it was his to take somewhere else. AWS ran the attestation. AWS decided, on every request, whether his own code was allowed to touch his own secrets. Then he asked the question that started this article. How do I port this to another provider? ...
Two postures have hardened around the cost of AI, and most leaders have already picked one without registering it as a choice. The first says zero dollars per token. Own the silicon, run the weights locally, drive the marginal cost of a query to nothing. Apple’s M3 through M5 put a capable model on a machine that fits in a backpack, NVIDIA’s GB10 desktop box puts a small token factory under the desk, and the appeal is clean: no meter, no vendor, no bill that grows every time the team does its job. ...
Every team trying to wring the best economics out of a serving stack is reaching for the same lever, and a lot of them are falling for the same trap. It does not matter whether the model lives in a single box (the 8-way B300 nodes reaching general availability right now carry roughly 2.3 TB of HBM) or spans an NCCL fabric across many of them: context and concurrency are capped by the KV cache, so KV-cache optimization is where everyone goes looking for free capacity. When a library advertises more KV tokens at no cost, it lands on the most expensive bottleneck in the deployment. The instinct to take that offer is correct. The discipline to measure it before you build a capacity plan on it is the part this note is about. ...
Yesterday a code editor sold for sixty billion dollars. SpaceX exercised an option it had struck back in April. The terms were unusually clean: buy Anysphere, the company behind the Cursor editor, outright for $60 billion in stock, or walk away and pay $10 billion just to partner. It bought. CBS reported the deal the same week SpaceX went public. Cursor leans heavily on Anthropic’s models today, and the new owner has already said it will drop its own models and Grok’s coding agent into that seat. ...
Everyone reaches for the GPU. At 114 megabytes, the GPU might be the wrong answer. This is the companion bench to Visual RAG Beats the Vision Model, where a distilled image-retrieval corpus beat a 3-billion-parameter vision model on accuracy at 180 times the speed. That whole argument rests on one quiet assumption: that the retrieval underneath is cheap. Cheap to run, cheap to host, cheap in watts. This article is where I check that assumption instead of asserting it, because an assumption you have not measured is just a hope wearing a lab coat. ...
A three-billion-parameter vision model looked at a reCAPTCHA tile and got it right 89 percent of the time. It took 128 milliseconds. A lookup over a few hundred megabytes got it right 95 percent of the time. It took seven-tenths of a millisecond. Same tiles. Same held-out set. One of those is how almost everyone is wiring computer vision into their stack this year. The other is how you should. ...
A reproducible, independent quality-and-throughput study of FP8, INT4-AWQ and NVFP4 against BF16 — across two dense and two Mixture-of-Experts models, measured with no access to NVIDIA’s harness. Reproduce it yourself. Every number below traces to a committed run log, and the entire pipeline is public and MIT-licensed: github.com/sch0tten/nvfp4-benchmark. Clone it, run make all, dispute a number, add a model — see §3.7. Abstract We benchmark four numeric formats — BF16, FP8, INT4-AWQ and NVFP4 — across sixteen arms (two dense and two Mixture-of-Experts instruction-tuned models, each in all four formats) on a single 96 GB NVIDIA Blackwell workstation, using the most-downloaded real-world quantization of each model rather than idealized in-house ones. On quality — measured generatively under one identical protocol with the EleutherAI harness — four bits is nearly free: averaged over five tasks, NVFP4’s cost is at most 0.6 points (the dense models) and the MoE models give up even less, and that cost is concentrated almost entirely in knowledge (MMLU-Pro); math, code and instruction-following sit at a ceiling. NVFP4 and INT4-AWQ are a wash at equal ~½ byte per parameter — which one wins is decided by the quantization recipe, not the number format. On throughput in the single-stream regime, the dominant lever is architecture: the MoE arms decode 3–7× faster than the dense ones, and within a model INT4-AWQ’s mature kernels usually edge NVFP4 on decode while NVFP4 holds the smallest weight footprint. With no access to NVIDIA’s harness, our independently-measured BF16→NVFP4 deltas reproduce NVIDIA’s published deltas to within 0.6 points on three of four benchmarks — and to 0.03 on the Qwen-MoE. The practical verdict for a local agentic deployment: run a 4-bit MoE; take INT4-AWQ for peak tokens-per-second today and the official NVFP4 for the smallest memory and the format Blackwell was built around. ...
In September 2006, a Debian maintainer did everything right and broke the world’s trust for a year and a half. He was cleaning up the OpenSSL package. Valgrind and Purify, the memory checkers every careful engineer is supposed to listen to, kept flagging two lines in md_rand.c. The lines read uninitialized memory. That’s a sin. Undefined behavior, the kind of thing you delete without a second thought. So he deleted it. ...