Charles Piller
π€ SpeakerAppearances Over Time
Podcast Appearances
Journals do not want replicated studies, especially if they're equivocal or if they disprove something, particularly if the original experiment showing something ostensibly important was done by a well-known or famous or perhaps just esteemed experimenter.
I can imagine if you're trying to grow your career and you're looking to have more citations, it's not helpful to do a replication study because odds are that it's not going to have a lot of citations. The original work might, but yours probably won't. Odds are you're not even going to get it published. Yeah.
I can imagine if you're trying to grow your career and you're looking to have more citations, it's not helpful to do a replication study because odds are that it's not going to have a lot of citations. The original work might, but yours probably won't. Odds are you're not even going to get it published. Yeah.
I can imagine if you're trying to grow your career and you're looking to have more citations, it's not helpful to do a replication study because odds are that it's not going to have a lot of citations. The original work might, but yours probably won't. Odds are you're not even going to get it published. Yeah.
Do you think that creates a crisis for us where we're putting out new research that is hellbent on creating a new discovery, but it's based on shaky science as we currently have it?
Do you think that creates a crisis for us where we're putting out new research that is hellbent on creating a new discovery, but it's based on shaky science as we currently have it?
Do you think that creates a crisis for us where we're putting out new research that is hellbent on creating a new discovery, but it's based on shaky science as we currently have it?
Well, let me say that I think that the vast majority of scientists are honest. They're trying their best to do something useful. They may feel the pressure of the field they're in and the competition and the incentive structures that may be misguided, but they're not dishonest. They're not cheating. They're not falsifying information. That said, there are many who are.
Well, let me say that I think that the vast majority of scientists are honest. They're trying their best to do something useful. They may feel the pressure of the field they're in and the competition and the incentive structures that may be misguided, but they're not dishonest. They're not cheating. They're not falsifying information. That said, there are many who are.
Well, let me say that I think that the vast majority of scientists are honest. They're trying their best to do something useful. They may feel the pressure of the field they're in and the competition and the incentive structures that may be misguided, but they're not dishonest. They're not cheating. They're not falsifying information. That said, there are many who are.
Just like in every walk of life, you have people who are cutting corners or doing something illegal or dishonest. And so I would say that people can trust the vast majority of experimentation.
Just like in every walk of life, you have people who are cutting corners or doing something illegal or dishonest. And so I would say that people can trust the vast majority of experimentation.
Just like in every walk of life, you have people who are cutting corners or doing something illegal or dishonest. And so I would say that people can trust the vast majority of experimentation.
I think what's not as well understood is that a few experiments, or a relative few of experiments, that are done improperly, that are based on false ideas, or are deliberately doctored, deliberately changed in inappropriate ways to, say, support a hypothesis that can't be supported by the actual data in the experiment, When those things happen, they can skew thinking in the field.
I think what's not as well understood is that a few experiments, or a relative few of experiments, that are done improperly, that are based on false ideas, or are deliberately doctored, deliberately changed in inappropriate ways to, say, support a hypothesis that can't be supported by the actual data in the experiment, When those things happen, they can skew thinking in the field.
I think what's not as well understood is that a few experiments, or a relative few of experiments, that are done improperly, that are based on false ideas, or are deliberately doctored, deliberately changed in inappropriate ways to, say, support a hypothesis that can't be supported by the actual data in the experiment, When those things happen, they can skew thinking in the field.
They can have subtle or sometimes very obvious effects in steering other scientists in certain directions, particularly if they're done by important people who have a lot of influence in the field, including some that I've written about.
They can have subtle or sometimes very obvious effects in steering other scientists in certain directions, particularly if they're done by important people who have a lot of influence in the field, including some that I've written about.
They can have subtle or sometimes very obvious effects in steering other scientists in certain directions, particularly if they're done by important people who have a lot of influence in the field, including some that I've written about.
How does that factor in when we look at our hierarchy of evidence and expert opinion being low on that pyramid, and as you go higher to the meta-analyses, the systemic reviews, how does it factor in that there are perhaps studies that were done with poor data, falsified data? How does that impact our ability to do those high-quality reviews?