Aye Moah

with a conversion button and then the page that didn't get traffic anymore.

So we are trying to basically get our get back out of that trouble, right?

But it's not like one of the major pages, but sometimes you don't always think of all the second order effect that can come through.

And it's really important to make sure what are your guardrail metrics and make sure that you are looking out for them.

No peeking should be self-explanatory.

It's rational.

When the experiment's in flight, don't look ahead and react or change things.

But it's really hard to have self-control.

Rationally, we know that.

But then when it's actually there, everybody want to peek.

Somebody from my team is here.

They have done it too.

So the other one is big swings, like the biggest effort.

don't necessarily bring the largest results.

Our most ambitious experiments, where we change how a product works at a one funnel step in a major way, didn't really bring the most valuable results to the company.

The gains were tiny, incremental, and we spent two and a half months building something, and it didn't even bring the same results as putting a big red button somewhere.

So sometimes just adding a red button might give you an extra 7% in conversion.

And I want to put a note on how to start this experimental culture.

I want to do a quick poll on where you are.

You are all well optimized across the entire funnel.