Ryan Peterman

The Peter Aronow book, I think, is probably as good as like a thing you could ever read if you're like a social scientist or someone working more practically.

It's a little less math heavy than the Wasserman books.

I mean, ironically, it's a great example of sort of everything we talked about today summed up as like a story half when we were trying to get Deltoid 3 to ship out.

You know, at that point, Deltoid 1 was still the default.

And there was a person who came to us.

And this person was actually like, oh, the way is great.

And I loved interacting with them.

But this interaction was really, I was like, oof, this reflects a lot of cultural pathology at our company where they're like, hey.

I can't let you ship Delta 3.

And I'm like, why?

Why can't we ship it?

And they're like, our holdout goes from being statistically significant win for the company to being not statistically significant than Delta 3.

And I think it's a regression.

And I was like, all right, it might be a regression, but maybe it's also the truth.

They're like, I actually don't know which it is, but you guys are going to wait until the half is over and we've decided we hit our goal and then you can ship Delta 3.

And I was like, oh, but...

Wait, I was like, your win is so close to the border between you did nothing and won that literally like mild tweaks in our code have turned it off.

I was like, I feel like that's not a win people would be so concerned about, and especially this notion that, like, you're almost significant, so therefore you failed, or you're just barely not significant, you know, you win or fail.

That, I think, is a super dangerous culture.

I mean, one of the worst things I ever saw was a team that I managed to squeeze, and one of the things they told me was they're like, hey, you know, we have to do this thing because we made a goal.