MLOps and Model Serving Latency

MLOps in production: serving small models on local GPUs, inference latency budgets, CUDA graphs, fallbacks, and the loop from training data to endpoint.

Performance Engineering Is the A in CIA

Every agentic pipeline we have designed this year has the same shape: a chain of serial hops, and somewhere in the chain, a call to a hosted API that does one thing better than anything you could build in a week. The budget conversation is always about the model at the end of the chain. The availability conversation, when it happens at all, is about whether the API is up. Nobody asks what the API costs on a healthy day, per turn, forever, and nobody files the answer under security. ...