The Forget The Funnel Podcast

Your A/B Tests Suck: The Key to Running Effective Experiments

12 Aug 2024

Audio

Description

SaaS teams use the best practice of A/B testing like a crutch for marketing, messaging and product growth decisions. And it makes sense. If you think about it, A/B testing something relieves you of so much burden. Not sure which headline is gonna resonate? Test it! Teammates have an (ahem, questionable) idea for how to improve conversions? Test it!Here’s the problem with the “test everything” approach: The vast majority of A/B testing is void of customer understanding and designed to make incremental improvements, typically without impacting revenue in any meaningful way.If you can 'cut the cord’ from the purely data-driven statistical model of trying to optimize a page or user journey, you can build bigger, better, more powerful A/B tests that hit customers directly in the feels (and your ARR).On this episode of the Forget the Funnel Podcast, Marc Thomas from Podia shares why testing isn’t the best practice it’s cracked up to be. Georgiana and Claire share when running tests is valuable and why knowing how your customers think, feel, and behave helps you run more productive and profitable tests.Timestamps:1:04—Marc Thomas discusses A/B testing as a flawed “best practice” in SaaS. He explains why it isn’t always the answer and why getting close to your best customers is a better alternative.10:28—Georgiana discusses scenarios when teams might use A/B testing, like validating a hypothesis before taking a big swing. She also explores when a smaller conversion rate can actually be a good thing.15:07—Claire explains why testing for testing’s sake will not produce the desired results. Instead, effective testing starts with understanding your customers to form stronger hypotheses.18:11—Georgiana tells a story of how testing went south for one company when the team tried to find a suitable pricing model. A/B testing steered them wrong until they zeroed in on their best-fit, higher-value customers and what value meant to them.23:29—Georgiana talks about the value of identifying Jobs-to-be-Done for one company so that they can build the customer experience around their high-value customers and optimize for the right things for the right customer.27:25—Claire points out how major organizational change can come from a handful of conversations and when SaaS companies should run tests.As always, you can learn more about Forget The Funnel here: Read the Forget The Funnel Book The AI book companion that comes free with the book Check out Forget the Funnel’s website

Transcription

This episode hasn't been transcribed yet

Help us prioritize this episode for transcription by upvoting it.

0 upvotes

🗳️ Sign in to Upvote

Popular episodes get transcribed faster

Other recent transcribed episodes

Transcribed and ready to explore now

3ª PARTE | 17 DIC 2025 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

13:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

12:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

10:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

13:00H | 20 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

12:00H | 20 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

Comments

There are no comments yet.

Please log in to write the first comment.

Report any issue

The Forget The Funnel Podcast

Your A/B Tests Suck: The Key to Running Effective Experiments

This episode hasn't been transcribed yet

Other recent transcribed episodes

3ª PARTE | 17 DIC 2025 | EL PARTIDAZO DE COPE

13:00H | 21 DIC 2025 | Fin de Semana

12:00H | 21 DIC 2025 | Fin de Semana

10:00H | 21 DIC 2025 | Fin de Semana

13:00H | 20 DIC 2025 | Fin de Semana

12:00H | 20 DIC 2025 | Fin de Semana

Sign in to Audioscrape

Share this moment