The Pragmatic Engineer

Measuring the impact of AI on software engineering – with Laura Tacho

23 Jul 2025

Audio

Description

Supported by Our Partners•⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more.• Graphite — The AI developer productivity platform.—There’s no shortage of bold claims about AI and developer productivity, but how do you separate signal from noise?In this episode of The Pragmatic Engineer, I’m joined by Laura Tacho, CTO at DX, to cut through the hype and share how well (or not) AI tools are actually working inside engineering orgs. Laura shares insights from DX’s research across 180+ companies, including surprising findings about where developers save the most time, why devs don’t use AI at all, and what kinds of rollouts lead to meaningful impact.We also discuss: • The problem with oversimplified AI headlines and how to think more critically about them• An overview of the DX AI Measurement framework• Learnings from Booking.com’s AI tool rollout• Common reasons developers aren’t using AI tools• Why using AI tools sometimes decreases developer satisfaction• Surprising results from DX’s 180+ company study• How AI-generated documentation differs from human-written docs• Why measuring developer experience before rolling out AI is essential• Why Laura thinks roadmaps are on their way out• And much more!—Timestamps(00:00) Intro(01:23) Laura’s take on AI overhyped headlines (10:46) Common questions Laura gets about AI implementation (11:49) How to measure AI’s impact (15:12) Why acceptance rate and lines of code are not sufficient measures of productivity(18:03) The Booking.com case study(20:37) Why some employees are not using AI (24:20) What developers are actually saving time on (29:14) What happens with the time savings(31:10) The surprising results from the DORA report on AI in engineering (33:44) A hypothesis around AI and flow state and the importance of talking to developers(35:59) What’s working in AI architecture (42:22) Learnings from WorkHuman’s adoption of Copilot (47:00) Consumption-based pricing, and the difficulty of allocating resources to AI (52:01) What DX Core 4 measures (55:32) The best outcomes of implementing AI (58:56) Why highly regulated industries are having the best results with AI rollout(1:00:30) Indeed’s structured AI rollout (1:04:22) Why migrations might be a good use case for AI (and a tip for doing it!) (1:07:30) Advice for engineering leads looking to get better at AI tooling and implementation (1:08:49) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:• AI Engineering in the real world• Measuring software engineering productivity• The AI Engineering stack• A new way to measure developer productivity – from the creators of DORA and SPACE—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

Transcription

This episode hasn't been transcribed yet

Help us prioritize this episode for transcription by upvoting it.

0 upvotes

🗳️ Sign in to Upvote

Popular episodes get transcribed faster

Other episodes from The Pragmatic Engineer

Transcribed and ready to explore now

Why Rust is different, with Alice Ryhl

20 May 2026

The Pragmatic Engineer

TypeScript, C# and Turbo Pascal with Anders Hejlsberg

13 May 2026

The Pragmatic Engineer

Designing Data-intensive Applications with Martin Kleppmann

22 Apr 2026

The Pragmatic Engineer

View all episodes from The Pragmatic Engineer

Comments

There are no comments yet.

Please log in to write the first comment.

Report any issue

The Pragmatic Engineer

Measuring the impact of AI on software engineering – with Laura Tacho

This episode hasn't been transcribed yet

Other episodes from The Pragmatic Engineer

Why Rust is different, with Alice Ryhl

TypeScript, C# and Turbo Pascal with Anders Hejlsberg

Designing Data-intensive Applications with Martin Kleppmann

Sign in to Audioscrape

Share this moment