Skip to content
Scan a barcode
Scan
Paperback AI Agent Evals: Evaluate and improve Single and Multi Agent Systems: A practical guide for PMs and AI engineers Book

ISBN: B0HQ5VM2KH

ISBN13: 9798167367586

AI Agent Evals: Evaluate and improve Single and Multi Agent Systems: A practical guide for PMs and AI engineers

AI agents do more than answer questions. They issue refunds, update records, and hand work to other agents, and a transcript that looks perfect can hide a refund that happened twice.

AI Agent Evals is a practical field guide to evaluating agents by what they changed in the systems they touch. Its central idea, the Evidence Ladder, gives product managers and AI engineers one rule they can apply in any design review: an agent should act on its own only up to the highest level of proof you can verify automatically.

The book follows one agent from start to finish: Speedy, the customer support assistant at Pronto, a fictional online grocery store. Every chapter tests one of Pronto's real policies, from the $50 refund rule to substitution opt-ins and safety escalations.

You'll learn how to:

Write task contracts that define success, including correct refusals and escalationsFind what breaks with error analysis, before choosing any metricBuild LLM judges, then grade them against expert labelsGrade tool calls and trajectories without dictating the pathRed-team boundaries, permissions, guardrails, and data leakageEvaluate RAG, memory, handoffs, and multi-agent systemsMonitor production, catch drift, and gate releases in CI/CDChoose between large and small models, and route between them, on evidencePromote an agent's autonomy only as far as the evidence supportsMake the release call, ship, limit, or hold, with a gate table written before the run

Every example is runnable Python, and a companion repository on GitHub runs the whole book end to end.

Recommended

Format: Paperback

Temporarily Unavailable

We receive fewer than 1 copy every 6 months.

Save to List

Customer Reviews

0 rating
Copyright © 2026 Thriftbooks.com Terms of Use | Privacy Policy | Do Not Sell/Share My Personal Information | Cookie Notice | Cookie Preferences | Accessibility Statement
ThriftBooks ® and the ThriftBooks ® logo are registered trademarks of Thrift Books Global, LLC
GoDaddy Verified and Secured