What happens when a seven-person data team restarts an experimentation program from scratch — and gets the whole company running experiments within months?Welcome to the first episode of our special miniseries on product experimentation. Now that you can build almost anything faster than ever, what matters most is learning what to build, what to keep, and what to put back on the shelf. Over the next few episodes, we go beyond A/B testing basics to explore how teams build experimentation culture, move faster with data, and make smarter product bets in practice.Hosting this series are two of Spotify’s finest experimentation nerds: Luke Frake, group product manager for subscriptions, and Mårten Schultzberg, staff data scientist for Spotify’s experimentation platform, Confidence (confidence.spotify.com).In this first episode, we head outside Spotify to explore how Huel, the UK-founded nutrition brand, approaches experimentation across a rapidly growing omnichannel business. Unlike pure digital products, Huel sells direct-to-consumer on its own site, on Amazon, in supermarkets, and on TikTok Shop — which means you can't always rely on a classical A/B test to measure what's working.Luke and Mårten are joined by Bhav Patel, Data Director at Huel, to dig into how his lean team rebuilt Huel's experimentation culture from the ground up. They discuss why Huel lets experiment ideas come from anyone — trading managers, marketing, even Bhav himself — and how a founder-led, data-first culture drove strong adoption of experimentation across the entire organization. You'll hear how Bhav's team tackles the unique challenges of omnichannel measurement: building synthetic control groups to evaluate TV campaigns, running geo-lift tests across countries, and stitching together fragmented user data across platforms that guard their ecosystems. The conversation also covers why relevancy matters more than experiment velocity, how AI is fast-tracking experiment design, and more.If you care about building better products with data, check it out — and stay tuned for more episodes in this series dropping soon.Learn more about how we experiment at Spotify:Dive deep into experimentation nerdiness on the Confidence blog👉 Here’s a good post to start with: “Beyond Winning: Spotify's Experiments with Learning Framework”Get started with Spotify’s product experimentation platform: Confidence Become an experimentation expert with our free experimentation bootcampFollow the Confidence team on LinkedIn for product updates, event announcements, experimentation insights, and more👋 Follow Luke on Linkedin👋 Follow Mårten on LinkedinRead what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.comYou should follow us on Twitter @SpotifyEng, LinkedIn, and YouTube!

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Transcript

Related