Adam Kucharski

I mean, actually, even the first randomized controlled trial in modern medicine, which is 1947, so streptomycin, a trial for TB, Austin Bradford Hill, who led that,

made the point that actually streptomycin had some very promising looking lab data and kind of early signals and he suggested it would have been unethical to withhold it from patients if it was available but actually it was 1947 there were currency controls the the uk in its post-war state couldn't get enough dollars to buy streptomycin so there wasn't enough to go around so in that situation they said it would be ethical to randomize because there's not enough of it so there's not enough of it you might as well randomize and just learn something along the way

And I think we've seen that in other situations.

I mean, in other sort of examples that you see where things are very difficult to randomize, you can think about natural experiments.

A lot of the well-known one is the Vietnam draft, where people essentially randomly assigned to go to war based on their birthdays.

A lot of economists have done Nobel Prize winning work.

Using that to understand the effects of war on subsequent life outcomes, because it's not something where you can fully design that experiment, but you can then make use of what you have available.

So I think a lot of it just comes down to this.

issue of we want to understand cause and effect and the benefit of randomization is a lot of the other things that would influence whether or not you know someone's getting a vaccine someone's getting the disease because you're randomizing on the vaccine on average those will cancel out as effects so it gives you that that quite neat benefit but of course you've also got the challenge that you might run a population in one group when one population that doesn't generalize to somewhere else you've also got the time issue so for diseases that evolve

You might run a trial now against flu or COVID or something.

A year later, that's going to be a different variant.

To what extent can you carry over those conclusions?

I think we see a lot of examples in the literature where, for instance, someone might run a trial in one population for one disease, for flu, for example, and then see a very different result when people look at population patterns elsewhere because it's a different immune structure, it's a different strain, it's a

Yeah, I think we can't just say, well, that study from a few years ago is the gold standard.

We're only going to use that one.

We have to think about how these things move along.

I mean, that being said, though, I think in COVID, there were missed opportunities, I think, to gather much stronger data.

I think it's very hard to justify running those kinds of studies as a threat increases.

I think when epidemics going up.

Taking your time to kind of try and randomize.