What happens when an AI agent follows your documented process perfectly, but that process bears little resemblance to how decisions are actually made? In this episode, I speak with Bill Wilson, Executive Head of Data and AI Solutions at NTT DATA UK&I. Bill oversees AI globally for NTT DATA's public sector work, giving him a close view of how governments are using AI while trying to manage risk, accountability, public confidence, and constrained resources. Bill offers a refreshingly practical test for any proposed AI system: is it competent, and what is the worst thing that could go wrong? He describes this potential consequence as the system's "blast radius." An AI assistant helping somebody understand a grant application presents a very different level of risk from an agent making decisions that affect employment, justice, taxation, or access to public services. We also discuss why companies can make a mistake before deploying their first agent. Automating an inefficient process simply allows the organization to perform the wrong work faster. Bill argues that teams should examine complete workflows, identify where several AI capabilities could produce a measurable result, and remain prepared to redesign the process as they learn. Another major problem is tacit knowledge. Employees frequently make decisions using experience that was never written down. An agent trained solely on formal documentation may therefore understand the official process while missing how the work gets done in practice. Bill explains how targeted questions, behavioral traces, feedback, and supervised learning could capture some of that reasoning. Public sector AI provides several useful examples. Bill discusses systems that process volumes of information beyond human capacity, emergency response work in Tennessee, and case management applications that gather information before a human reviews it. In these situations, AI can reduce administrative work and waiting times while leaving consequential decisions with people. But human approval alone provides no guarantee. If employees lose direct experience of the work, they may eventually approve whatever the system recommends. Bill compares this with airline pilots maintaining manual flying skills and describes how known test cases can reveal when reviewers are becoming overly trusting. For CIOs deciding which AI pilots should reach production, the advice is equally direct: choose work with measurable returns, group related use cases where their combined effect can be seen, learn from a varied set of deployments, and avoid building something a software provider is about to include in an existing product. As AI agents gain access to external information, internal data, and operational tools, how should your organization decide what they may do alone and when a person must intervene? Listen to the conversation and share your thoughts with me.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Transcript

Related