Laure Wynants

Another thing is that you need a lot of data.

So the case control...

It could work, but you would need to have to adjust in a statistical way for the fact that you use case control sampling.

And then still you would need representative controls.

They cannot be controls from before the outbreak or from healthy people.

So it brings a lot of complex issues.

The fact that we don't have good quality data yet, or we didn't at the time that these models were developed,

That's where a lot of the problems we saw have arisen.

Well, the first thing that struck us was that there's a lot of models out there already.

So we have 27 models identified.

And the most important finding, unfortunately, was that they were all at high risk of bias.

So that means that they are not ready to be applied in practice yet.

And that risk of bias comes from all sorts of problems, including non-consecutive patients or patient series that are not representative of the target population, and also the too complex modeling and too small data sets.

Another thing that should be stressed is the need for rigorous validation.

We realize that when you build models like this, it may not always be possible to validate them externally, but still then, if you have developed a model, you cannot just test it in the data set that was used to build it.

You need to bootstrap validation or cross-validation or leave center out cross-validation.

to have a good indication of how well this model would work if you would apply it in new patients.

With respect to validation, it's also important that researchers report their model.

So, for example, give the full equation behind the model or provide a software object.

And we have seen that 10 out of the 27 studies that we included didn't do that.