Mary Childs

So after a few months, Abel sends his neutral-toned official email to the authors of the paper that Felix and his team had replicated in Montreal, saying that the code had worked, but that they found the results don't hold up.

And when it came to the bigger problem, the fact that their results had fallen apart when the replicators removed that one cartel?

They say they set out to investigate if the cartel Los Zetas had changed the types of crimes they did after the war on drugs.

And their papers succeeded at proving that.

To be blunt, it doesn't make any sense.

That is Paolo Pinotti, a professor also at Bocconi University.

He said it was like doing a study on the effect of spreadsheets on productivity and then saying, oh, but the results don't hold up if you exclude Microsoft Excel.

Mary, when we first rocked up to the replication games back in May, I think we were both excited at the idea that we might watch some junior economists uncover some major problem with a published paper in real time.

But Abel had a different take when we asked him about the problems that the teams there had uncovered.

Like the team, for example, that had found issues in the government trust paper.

That seems like success.

Success.

It depends what you define as success.

Well, the process working as it's supposed to.

I mean, in a world in which science works, I think this should have been picked up before it's published, cited, and disseminated.

So I don't think it's a success.

That's fair.

The journal declined to comment, though they said they have a robust process to investigate concerns.

To me, this is a failure of the system, which is fine.

There's always going to be failures.