Kalyanarangan-HI1200×1505 106 KB

Vivek Kalyanarangan

If you’ve worked with modern AI models in production, you’ve probably run into the same wall: great results in development, followed by uncomfortable conversations about memory, latency, and cost. Quantization and Fast Inference is built for that exact point in the workflow.

This book walks through how to shrink and speed up models without redesigning them from scratch. It starts with the fundamentals—what quantization actually does to numbers and why it works—then moves into techniques you can apply right away. You’ll get hands-on with post-training quantization (PTQ), quantization-aware training (QAT), and the details that tend to cause trouble in practice, like activation outliers in LLMs or pressure on the KV cache.

What stands out is the full pipeline view. It doesn’t stop at “here’s how to quantize a model.” It covers how those choices affect deployment, runtime behavior, and tradeoffs you have to make along the way. There’s also coverage of newer low-precision formats like NF4 and FP4, which are starting to show up more often in real systems.

If you’re trying to run larger models on tighter budgets—or just want to understand what’s happening under the hood when you compress them—this is a solid place to dig in while the book is still in Early Access.

• Full details: Quantization and Fast Inference - Vivek Kalyanarangan

Don’t forget you can get 45% off with your Devtalk discount! Just use the coupon code “devtalk.com” at checkout