Show notes
Summary: Tobi challenged marketing’s fixation on prediction. He has built highly accurate LTV models, but accuracy alone does not move revenue. Marketing is intervention. Correlation shows patterns; causality tells you what happens when you pull a lever. That shift reshapes experimentation, explains why dynamic allocation can outperform static A B tests, and highlights how self learning systems can backfire or get stuck in local maxima. It also fuels his skepticism of unleashing agentic AI on historical data without a causal layer. If you want to change outcomes instead of forecast them, your systems need to understand levers and log decisions you can actually audit.(00:00) - Intro
(01:22) - In This Episode
(04:07) - Why Predictive Models Fail Without Causal Inference
(09:49) - How to Validate Causal Impact on Customer Lifetime Value
(13:04) - Reducing Uncertainty Around Causal Effects by Optimizing Levers, Not Labels
(17:01) - Why Dynamic Allocation Works Better Than Fixed Horizon A B Testing
(31:54) - The Boomerang Effect and Why Uninformed AI Sabotages Early Results
(40:15) - Escaping Local Maxima and The Failure of Randomly Initialized Decisioning
(44:04) - Why Agentic AI Trained on Data Warehouse Correlations Reinforces Bias
(49:00) - The Power of Composable Decisioning
(53:06) - How Machine Decisioning Transcends Marketing
(01:01:41) - Why Clear Priority Hierarchies Improve Executive Decision Making
About TobiasTobias Konitzer, PhD is VP of AI at GrowthLoop, where he’s chasing closed-loop marketing powered by reinforcement learning, causality, and agentic systems. He’s spent the past decade focused on one core problem: moving beyond prediction to actually influencing outcomes.Previously, Tobi was Chief Innovation Officer at Fenix Commerce, helping major eCommerce brands modernize checkout and delivery with machine learning. He also founded Ocurate, a venture-backed startup that predicted customer lifetime value to optimize ad bidding in real time, raising $5.5M and scaling to $500K+ ARR before its acquisition. Earlier, he co-founded PredictWise, building psychographic and behavioral targeting models that drove over $2M in revenue.Tobi earned his PhD in Computational Social Science from Stanford and worked at Facebook Research on large-scale ML and bias correction. Originally from Germany and based in the Bay Area since 2013, he writes frequently about causal thinking, machine decisioning, and the future of marketing.Why Predictive Models Fail Without Causal InferencePrediction dominates most marketing roadmaps. Teams invest months refining churn models, tightening confidence intervals, and debating which threshold deserves a campaign. Tobi built an entire company on that logic. His team produced highly accurate lifetime value predictions using deep learning and granular event data. The forecasts were sharp. The lift curves were clean. Buyers were impressed.Then lifecycle marketers asked a more uncomfortable question: what action should follow the score?A predictive model encodes the current trajectory of a customer under existing policies. It describes what will likely happen if nothing changes. Marketing changes things constantly. The moment you intervene, you alter the system that generated the prediction. The forecast reflects yesterday’s conditions, not tomorrow’s strategy.> “Prediction tells you the future if you do nothing. Causation tells you how to change it.”Consider the Prediction Trap.On the left, the status quo labels a person as high churn risk. The function is observation. The outcome is a description of what happens if you leave the system untouched. On the right, a lever gets pulled. The function is intervention. The outcome is directional change.That shift in function changes how you work.Prediction thinking centers on segmentation:Who is likely to churn?Who is likely to buy?Who looks like high LTV?Causal thinking centers on levers:Which incentive reduces churn?Which sequence increases repeat purchase?Which offer raises lifetime value incrementally?Tobi often uses an LTV example to expose the trap. Suppose high LTV customers frequently viewed a specific product early in their journey. A team might redesign the onboarding flow to feature that product more aggressively. The correlation looks persuasive. The causal effect remains unknown.Several alternative explanations could drive the pattern:The product may correlate with a specific acquisition channel.The product may have been highlighted during a limited campaign.The product view may signal prior brand familiarity.Only an intervention test can estimate incremental impact. Correlation can guide hypothesis generation, but it cannot validate the lever itself.Tobi also highlights a deeper issue. Acting on predictions introduces compounding uncertainty across multiple layers:The predictive model carries statistical variance.The translation from model features to campaign strategy introduces interpretation bias.The experiment introduces sampling error.Execution introduces operational noise.Each layer adds variability. When teams treat prediction accuracy as the goal, they lose visibility into where uncertainty enters the system. When teams focus on intervention impact, they concentrate measurement on the lever that drives revenue.Boardrooms already operate in causal language. Incremental ROI is causal. Budget allocation is causal. Executives care about what caused growth, not which segment looked promising in a dashboard. Prediction can inform prioritization. Causal inference determines what to scale.If you want to move in that direction, adjust your operating model:Start every initiative with a controllable lever.Define the action before defining the segment.Design experiments that isolate the incremental effect of that lever.Randomized or adaptive allocation both estimate causal lift.Report impact in revenue, retention, or contribution margin.Tie every experiment to a business outcome.Document assumptions and uncertainty.Build institutional memory around what caused change.Prediction remains useful. Intervention drives growth. Teams that understand that distinction build systems that learn through action instead of watching the future unfold from the sidelines.Key takeaway: Anchor your marketing engine in causal experiments. For every predictive score, define the specific action it informs, test that action against a control, and quantify incremental lift tied directly to revenue or retention. Replace segment rankings with lever performance dashboards that show effect size, confidence, and business impact. When every campaign answers the question “What did this intervention cause?” your team shifts from observing trajectories to shaping them.How to Validate Causal Impact on Customer Lifetime ValueMost teams treat high LTV segments as proof of where to spend. The model ranks customers. The top decile looks profitable. Budget flows upward. Tobi described asking the head of CRM at a billion dollar outdoor brand what he does when a model predicts someone will be high LTV. The answer came instantly: Spend more on them, no?That instinct feels responsible. It also confuses observation with intervention. Introducing the high LTV Fallacy:On the right side of the chart, you see a dense cluster labeled high LTV customers. Revenue increases with marketing spend. The correlation line slopes upward. It looks clean and convincing. They were going to buy anyway. That cluster may represent customers with higher income, stronger brand affinit...