Adam Kucharski

But I think what that did is it created this impression that particularly epidemiology was driving the decision making.

lot more than it perhaps was in reality because so much of that was being made public and a lot more of the evidence around education or economics was being done behind the scenes.

I think that created this kind of asymmetry in public perception about how that was feeding in.

It's really hard as well as a scientist when you've got journalists asking you how to run the country to work out those steps of, am I describing

evidence behind what we're seeing am i describing the evidence about different interventions or am i proposing you know to some extent my value system on what we do and i think all of that in in very kind of intense times um can be very easy to get blurred together in in public communication i think we saw a few examples of that where you know things were being the follow the science on policy type angle where actually once you get into what you're prioritizing within a society quite rightly you've got other things beyond just the epidemiology driving that

Yeah I think that the story of of kind of how we rank evidence in medicine is a fascinating one I mean even just how long it took for people to think about these elements of randomization and fundamentally the what we're trying to do when we we have evidence here in medicine or science is prevent ourselves from confusing randomness for a signal I mean that's that's fundamentally you know we don't want to mistake something don't think it's going on it's not um

And the challenge, particularly with any intervention, is you only get to see one version of reality.

You can't give someone a drug, follow them, rewind history, not give them the drug, and then follow them again.

So one of the things that essentially randomization allows us to do is if you have two groups, one that's been randomized, one that hasn't, on average, the difference in outcomes between those groups is going to be down to the treatment effect.

It doesn't necessarily mean in reality that would be the case, but on average, that's the expectation that you'd have.

And it's kind of interesting, actually, that the first modern randomized controlled trial in medicine in 1947, this is for TB and streptomycin, the randomization element, actually, it wasn't so much statistical as behavioral.

If you have people come into hospital, you could...

some extent just say we'll just alternate we're not going to randomize we're just going to first patient we'll say is a control second patient treatment but what they found in a lot of previous studies was you know doctors have bias maybe that patient looks a little bit ill or that one yeah maybe is on borderline for eligibility and often you got these quite striking imbalances when you allowed it for human judgment so it's really about kind of shielding against those behavioral um

elements but I think there's a few situations it's a really powerful tool for a lot of these questions but as you mentioned one is this issue of you know you have the population you study on and then perhaps in reality how that translates elsewhere and we see I mean things like flu vaccines are a good example which are very dependent on immunity and evolution and what goes on in different populations sometimes you've had a result on a vaccine in one place and then the effectiveness doesn't translate in the same way to somewhere else I think the other really important thing to bear in mind is

as I said, it's the averaging that you're getting an average effect between two different groups.

And I think we see a certain lot of development around things like personalized medicine, where actually you want, you're much more interested in the outcome for the individual.

And so, you know, what a trial can give you evidence of is on average across a group, this is the effect that I can expect this intervention to have.

But we've now seen more of the emergence of things like N equals one studies where you can actually, you know, over the same individual, particularly kind of chronic conditions, look at those kind of interventions.

And also there's just these extreme examples where you're ethically not going to run a trial.

You know, there's never been a trial of whether it's a good idea to have intensive care units in hospitals.