Show notes
What’s up folks, welcome to our 4 part series of Crawling through the dungeon of martech architecture. You’ve arrived at Part 2: The Eye of Context.We cover:(00:00) - Intro
(00:56) - In This Episode
(01:28) - Sponsor GrowthLoop
(02:32) - Sponsor: GrowthBench
(03:32) - Welcome Back
(04:09) - FLOOR 2: THE EYE OF CONTEXT
(06:15) - Why AI Produces Believable Nonsense
(09:00) - BOSS BATTLE: The Hallucination Oracle
(10:07) - Data Quality: When Agents Read Your Messy Data
(22:33) - Context Engineering: What It Is and Why It's Not the Same as Prompt Engineering
(24:28) - Sponsor: MoEngage
(25:25) - Sponsor: Knak
(26:30) - Context Eng vs Prompt Eng
(38:58) - Why the Industry Built the Wrong Semantic Layer in 2012
(46:33) - How Context Rot and Fragmentation Break AI Agent Performance
(49:59) - BOSS BATTLE: Rotten Context Mage
(50:37) - How to Build a Shared Context Layer for AI Agents
(58:17) - Testing Whether Your Context Layer Works
(01:01:35) - NEW ACHIEVEMENT: The Meaning Layer Is Live
---------------------------------------------------------------------------OPENING---------------------------------------------------------------------------Welcome back to the Dungeon of Martech Architecture.You’ve arrived at part 2. If this is your starting point, check out part 1 where we cleared the first floor’s boss in 2 forms: The False Truth King in the CRM, and The Export Hydra that spread it everywhere. That said, if you already have a data warehouse, you might be able to start right here.Episode 1: CRM GravityWe conquered the source of truth and discovered that the data warehouse replaces the CRM with portable audiences.Episode 2: The Eye of ContextToday, we learn why AI fails without shared meaning, build the context engineering layer, and dig into why the industry built the wrong kind of meaning infrastructure in 2012.Episode 3: The Correlation MasqueradeNext, we escape the correlation trap and build the causal memory layer that separates agents that optimize correctly from agents that confidently scale the wrong behavior.Episode 4: The Dispatch TowerThen, we tackle the governance chaos of 30 vendors all claiming authority, and confront the interface decision that most organizations already made without realizing it.Let’s start our descent.---------------------------------------------------------------------------FLOOR 2: THE EYE OF CONTEXT — AI Hallucinations, Data Quality, and Context Engineering---------------------------------------------------------------------------The layout of the second floor down the dungeon of martech architecture actually looks pretty fancy. It’s cozy, it looks modern, the whole palace is lined with mirrors. But it’s a bit creepy because once you look a little closer at the reflections, you notice that some of the details are off. The boss on this floor is low key danger that sneaks up on way too many teams – not like the big flashy monsters from the past 2 floors.Let’s say you have a new AI system running on your marketing data. You’ve got it producing stuff like scores, recommendations, campaign ideas. Initially, it actually looks solid.There’s no obvious AI sentence structures in the summaries, they read well.The scores next to accounts seem to make sense: higher ones next to well known brands and lower ones are gmail accounts.The campaign ideas are actually pretty fresh, you can tell that it’s tailored for your ICP.Every output is delivered with impeccable confidence.So the next step is asking yourself… how would you know if it was wrong?There’s a lot of obvious hallucinations that you probably catch when you chat with GPT or Claude, like totally inventing stuff. I’m talking about the details. The kind of wrong that passes the first glance. We’ve all seen that viral post on r/analytics about a company that found out AI has been making up analytics data for 3 months. Whether this post is from a real story or not, some versions of this are happening inside companies today. But this is happening everywhere right now. Even at some of the top AI companies on the planet. I’ve talked to technical marketing leaders that have greenlit agentic tools at their startups before the data definitions were settled. One of them called it ‘believable nonsense’, and that term kinda stuck with me. It’s the most dangerous form of hallucination, it sneaks up on you.This floor is harder than the last because the traps on the previous level were visible, once you knew what to look for: CRM exports nobody trusted, audience logic duplicated across platforms, copies drifting from the original. You felt that, you saw that. This floor’s failures are designed to look like success, until you dig into the details and look under the hood.Let’s look at the origins of The Hallucination Oracle boss. Why AI Produces Believable NonsenseHumans are wired to trust smooth, confident talkers. It’s actually baked into our evolution and how our brains develop from infancy. Studies on babies and brain scans show this is an innate thing that kicks in early.A person who sounds certain usually knows something, right?LLMs and AI systems break this calibration, they produce fluency without necessarily producing correctness. At first glance, the output sounds right for structural reasons, not evidential ones. And once something sounds right, we engage with it differently. We forward it. We build on it. We present it to stakeholders who don't have the context to question it.This is similar to the The False Truth King boss from the first floor in episode 1, but this is at the intelligence layer: output that has been processed, synthesized, and returned with all the structural markers of a trustworthy answer, but the problem is the reasoning underneath it is hollow.That’s Jason Dobbs, Head of Marketing & GTM Engineering at Kumo. He greenlit agentic analytics and predictive workflows at his startup before the team had settled on shared data definitions. The system produced outputs that looked reasonable right up until someone started asking follow-up questions:JASON DOBBS, Kumo AI"What made it dangerous wasn't really like the obvious hallucination. What we were seeing looked polished enough to be operational. At first glance the scores looked precise, the summary sounded coherent, the recommendations felt data-backed. But the moment you ask the simple follow-up questions -- why did you choose this account? What data drove this decision? -- the logic started to thin out. The lesson wasn't that the data was bad or the warehouse was bad or the model was bad. What was wrong is we were trying to automate ambiguity. We were asking AI to solve for confusion that we hadn't yet ourselves solved for internally. And once you do that, you enter the danger zone because the failure is essentially believable nonsense."This is some scary stuff right? When this believable nonsense gets trusted long enough to make it into a campaign, a decision, a board slide. Obvious hallucinations are easy to catch. Confident, polished, data-backed nonsense gets through more often than we think.The good news is that we already have a weapon perfect to slay this boss, we shaped it in the last episode: the data warehouse. It houses data. Data is how we defeat believable nonsense… but we need to enhance it.---------------------------------------------------------------------------BOSS BATTLE: The Hallucination Oracle----------------------------------------...