Skip to content
ISBN: B0G2FY1FDW
ISBN13: 9798274507356

Deepspeed in Production: INFERENCE OPTIMIZATION AND MODEL: Deploy LLMs efficiently with optimized serving, quantization, and low-latency inference for

$33.72List Price: $34.95
Save $1.23!
ISBN: B0G2FY1FDW
ISBN13: 9798274507356
Formats

Conditions

A brand-new, unused copy in perfect condition.

Formats

Conditions

A brand-new, unused copy in perfect condition.

See all 1 editionsfrom $33.72

Book Overview

Run large language models with predictable latency, controlled cost, and production reliability.

Shipping LLMs is an operational problem. Teams struggle with time to first token, tokens per second, GPU memory pressure, and a moving target of engines and datatypes. This book turns those issues into clear practices you can apply with DeepSpeed and the serving layers you already use.

You get a practical path from checkpoint to stable API, with configuration that fits real workloads, not toy demos. Every topic is grounded in measurable outcomes so your stack meets SLOs under mixed traffic and budget constraints.

  • place DeepSpeed correctly in your stack and configure kernel injection, tensor parallel, and ZeRO for real services
  • understand TTFT and throughput from prefill to decode and set metrics for p95 latency and queue time
  • size and control the KV cache with paged attention, batching, and safe headroom targets
  • apply quantization that holds up under load, including w8a8, awq, gptq, fp8, and fp4
  • use speculative decoding with a sound drafter choice, acceptance math, and stable fallbacks
  • operate vllm, tensorrt llm on triton, and tgi with clean api surfaces and core flags
  • scale with ray serve and plan capacity from workload shapes and arrival patterns
  • tune for nvidia hopper and blackwell or amd mi300x, with attention backends and nvlink planning
  • run on kubernetes with gpu operator, device plugin, mig, and topology aware placement
  • wire observability with prometheus, dcgm, and opentelemetry spans, plus vllm bench, trtllm bench, and genai perf
  • ship safely with quotas, redaction, audit logs, go live gates, and instant rollback plans

This is a code heavy guide with working YAML, JSON, Shell, and Python examples that map directly to production, from gateway limits and network policies to rollout templates and exportable benchmark scripts.

Grab your copy today and build an LLM service that stays fast, measurable, and dependable.

Format:
Paperback
Language:
English
ISBN:
B0G2FY1FDW
ISBN13:
9798274507356
Release Date:
November 2025
Publisher:
Independently Published
Length:
288 Pages
Weight:
1.12 lbs.
Dimensions:
0.6″ × 7.0″ × 10.0″
ComputersComputers & Technology

More by TARA MALHOTRA

Customer Reviews

0 customer ratings | 0 reviews
Write a Review

There are currently no reviews. Be the first to review this work.