Skip to content
Scan a barcode
Scan
Paperback Inference Engineering in Practice: Serving LLMs Fast and Cheap: Batching, KV Cache, Quantization, and the Cost of Every Token Book

ISBN: B0HKG4WJH2

ISBN13: 9798175785631

Inference Engineering in Practice: Serving LLMs Fast and Cheap: Batching, KV Cache, Quantization, and the Cost of Every Token

Sooner or later the question arrives in a thread somebody added you to on a Friday afternoon: could we serve this model ourselves, and what would it cost?

The honest answer is arithmetic, and it needs four things the question never supplies - which tokens you are counting, at what latency bound, at what utilization, and a throughput number that belongs to your hardware rather than to a launch blog. Leave one of them unstated and two competent engineers will price the same model a factor of ten apart, each correct about the number they actually computed.

Inference Engineering in Practice is that arithmetic, and the machinery underneath it: nineteen chapters for the engineer who owns an inference budget and a pager, and has to decide rather than admire.

The argument, in one paragraph. Inference is a budget with two currencies, latency and money, and every serving decision picks an operating point on the frontier between throughput per accelerator and speed per user. Some levers trade one currency for the other. Others push the frontier outward and charge you in a third - memory, accuracy, operational risk, or the chance that the answer quietly changes. Knowing which kind of lever is in your hand before you pull it is the job.

Six parts, in the order the work has to be done:

The numbers that decide - what a served token costs, why decode is slow for reasons you can read off a datasheet, SLOs for a token stream, and a benchmark that will not flatter you.The economics - cost per million tokens and its three traps, the self-host-or-API break-even and what keeps moving it, and sizing a fleet from a traffic forecast.The scheduler's levers - continuous batching and the knee where it stops paying, the KV cache as a budget you can overdraw, and prefix caching.The model's levers - quantization behind an accuracy gate you built yourself, speculative decoding and the concurrency where it collapses, and what changes once one GPU is not enough.Running the fleet - routing for the cache, the adapter and the tenant; the serving signals that lie to you; autoscaling, cold starts and overload.Keeping the answer right - the endpoint contract, release gates, determinism and what it costs, and what to do when an accelerator fails.

The last chapter turns all of it into a worksheet that produces a written inference plan you can defend line by line.

The rule that shapes every number in it. This book prints no GPU measurement of its own. Every hardware-bound figure is a published benchmark, a vendor datasheet, or arithmetic on one of those with the arithmetic shown; every measured number came off a 16 GB laptop and says so. So every GPU figure is a published benchmark you can check at its source, and every lab re-runs on your own laptop for nothing with a single command. A free public companion repository carries one runnable lab per chapter, pinned to exact versions.

Who it is for. Backend and platform engineers who ship production services, read Python, have carried a pager, and have been handed a model or a model's monthly invoice. Not a book about model architecture, training, fine-tuning, prompting or retrieval, and it will not teach you to write kernels.

Vendor-neutral on purpose: every mechanism is explained in words that survive an interface change, and only then is the current flag named, with its version and its date.

Volume 3 of The AI Engineer's Library, after Observability for LLM Applications and Agents in Production. Nineteen chapters, about 86,000 words, 50 figures and screenshots.

Recommended

Format: Paperback

Condition: New

$29.51
Save $0.48!
List Price $29.99
Ships within 2-3 days
Save to List

Customer Reviews

0 rating
Copyright © 2026 Thriftbooks.com Terms of Use | Privacy Policy | Do Not Sell/Share My Personal Information | Cookie Notice | Cookie Preferences | Accessibility Statement
ThriftBooks ® and the ThriftBooks ® logo are registered trademarks of Thrift Books Global, LLC
GoDaddy Verified and Secured