IO Storage & Data Path

Everyone reaches for the GPU. At 114 megabytes, the GPU might be the wrong answer. This is the companion bench to Visual RAG Beats the Vision Model, where a distilled image-retrieval corpus beat a 3-billion-parameter vision model on accuracy at 180 times the speed. That whole argument rests on one quiet assumption: that the retrieval underneath is cheap. Cheap to run, cheap to host, cheap in watts. This article is where I check that assumption instead of asserting it, because an assumption you have not measured is just a hope wearing a lab coat. ...

Three months ago, I plugged a Blackwell GPU into my lab bench and pointed it at a million PDFs. Corporate documents: contracts, engineering reports, compliance filings, vendor proposals, maintenance logs, insurance certificates. The kind of archive that accumulates over two decades of running critical infrastructure. A million files. Not sampled, not curated, not cleaned. Raw. The plan was straightforward: parse the documents, chunk them, embed them into a vector store, wire up a retrieval layer, and start asking questions no keyword search could answer. Retrieval-Augmented Generation. The acronym that launched a thousand vendor decks. ...

Right now, I’m working on an InfiniBand topology design for a GPU cluster. The math keeps pointing to the same conclusion: scale-out only makes sense when scale-in has topped out. It’s not about CUDA cores. It’s not about tensor throughput. It’s about tail latency. NVLink keeps GPU-to-GPU communication on-package or over short copper links — no NIC, no PCIe host traversal, no protocol stack. For small messages, that means sub-microsecond latency in the hundreds-of-nanoseconds range. InfiniBand NDR switches add sub-microsecond port-to-port latency, but once you include the full path — PCIe to the NIC, driver overhead, fabric hops, and back — real-world GPU-to-GPU latency across nodes often lands in the 3-10μs range depending on message size and topology. ...