Alexi Horowitz-Ghazi

So we have a win.

Yes, one.

And we have more to check.

A lot more.

But we got one.

That's great.

And if the paper passes the whole first phase, if the code does spit out all the answers that the author said it would...

Then the replicators move on to phase two, robustness checks.

For robustness check, we kind of like change some parts of the model to see whether the original conclusion is still kind of makes sense.

This phase is less objective and requires more context and thought.

It requires the economist to consider the questions that the paper authors didn't think of or didn't write about.

The decisions the authors made and the decisions they could have made but didn't.

It's like trying to see the negative space in and around the paper.

There are infinite potential choices that researchers make or don't make.

And the replicators have such limited time, so they're not going to be able to consider and analyze everything.

They're just going to get through as much as they can.

They recreated what should be the same data set following the instructions that the authors left.

They ran the code.

And then we visited the group looking at that paper about cartel behavior in Mexico.

That group has found something, too.