LLM KV Cache Quantization

What KV-cache quantization buys and costs when serving LLMs: how lower-precision keys and values trade memory budget against measured output quality.