Episode 49: Rethinking AI Agent Evaluations

episode

AWS re:Think Podcast not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

Related

recent transcribed audio

Your partner wants profit. You want freedom. Now what? 22 min · 16 Sep 2026 · The Optometry Success Podcast

Board of County Supervisors Meeting - Work Session - Sep 15, 2026 3h 17m · 15 Sep 2026 · Prince William County Board of Supervisors

Finance and Budget Committee - Sep 15, 2026 1h 43m · 15 Sep 2026 · Prince William County Finance and Budget Committee

The Art of True Happiness | Dr. Arthur Brooks 2h 49m · 14 Sep 2026 · Huberman Lab

The Chris Wolfe Podcast: The Power of Quality Supplements 48 min · 14 Sep 2026 · EyeCode Media

NPR News: 09-13-2026 6PM EDT 4 min · 13 Sep 2026 · NPR News Now

Report any issue

Source

Recorded17 Apr 2026

Publisherrss.art19.com

Handlerec:17908209

Show notes

In this episode we explore how companies should evaluate AI agents across multiple dimensions — including correctness, tool selection, multi-turn reasoning, and safety . The conversation covers building reliable evaluation frameworks, balancing automated vs. human-in-the-loop testing, and leveraging observability to debug agent behavior in production.Links from the ShowAgentCore Evaluation: https://github.com/awslabs/agentcore-samples/tree/main/01-tutorials/07-AgentCore-evaluationsStrands Evaluation: https://strandsagents.com/docs/user-guide/evals-sdk/quickstart/AWS Hosts: Nolan Chen & Malini ChatterjeeEmail Your Feedback: [email protected]

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Sign in