Tobias "Tobi" Konitzer

You're paying real money with this 50-50 split.

Because you're withholding the good treatment that you know is going to lift the damn thing by 10%.

You're withholding that from 50% of your people in perpetuity.

So how can you get away with a split that still gets you to stat sick while also maximizing the ROI from the experiment?

Maybe it's a 1090 split.

Maybe it's a 991 split.

But in order to do that forecasting, you need to see all the data.

Guess what?

We're the only player that sees all the data.

That was one pull.

The other pull is what we call always-on measurement, which is...

The marketing conundrum.

It's related, but different.

So you run an experiment, you observe the winner, right?

One condition is clearly winning on the metric that you care about.

Scale it up, right?

Bam, 100% of the traffic to this thing.

Okay, but now you can't measure ROI anymore.

So now your new boss comes in two years later and says, you know what, Noah, you've done this thing two years ago.

You told everybody, I found your PowerPoint slides, right?