Brian Nosek
speaker
65 appearances
2 recordings
1 series
first heard Dec 2024
last heard Jan 2025
Brian Nosek’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
Our hypotheses are always right. And of course, it's laughable because we know that's not what's actually happening. We know that all that failed stuff is getting left out and we're not seeing it. And the actual literature is an exaggeration of what the real literature is.
I think there is broad buy-in on the need to change, and it has already hit the mainstream of many of the changes that we promote, sharing data, materials, code, pre-registering research, reporting all outcomes. So we're in the scaling phase for those activities, and what I am optimistic about is that we
There is this meta-science community that is interrogating whether these solutions are actually having the desired impact. And so this is the most exciting part of the movement as I'm looking to the future is this dialogue between activism and reform. We can do these things. Let's make these changes. And meta-science and evaluation. Is this working? Did you do what you said it was going to do?
And et cetera. And I hope that the tightness of that loop will stay tight because that, I think, will make for a very healthy discipline that is constantly skeptical of itself and constantly looking to do better.
There really is accelerating movement in the sense that some of the base principles of we need to be more transparent, we need to improve data sharing, we need to facilitate the processes of self-correction are not just head nods, yeah, that's an important thing, but have really moved into, yeah, how are we going to do that?
And so I guess that's been the theme of 2024 is how can we help people do it well?
One of the more exciting things that we've been working on is a new initiative that we're calling Lifecycle Journal. And the basic idea is to reimagine scholarly publishing without the original constraints of paper. A lot of how the peer review process and publishing occurs today was done because of the limits of paper.
But in a world where we can actually communicate digitally, there's no reason that we need to wait till the research is done to provide some evaluation. There's no reason to consider it final when it could be easily revised and updated. There's no reason to think of review as a singular one set of activities that by three people who judge the entire thing.
And so we will have a full marketplace of evaluation services that are each evaluating the research in different ways. It'll happen across the research lifecycle from planning through completion. And researchers will always be able to update and revise when errors or corrections are needed.
My whole life is about trying to promote transparent research practices, greater openness, trying to improve rigor and reproducibility. I am just as vulnerable to error as anybody else. And so one of the real lessons, I think, is that without transparency, these errors will go unexposed.
It would have been very hard for the critics to identify that we had screwed this up without being able to access the portions of the materials that we were able to make public. And as people are engaged with critique and pursuing transparency – and transparency is becoming more normal –
We might, for a while, see an ironic effect, which is transparency seems to be associated with poorer research because more errors are identified. And that ought to happen because errors are occurring. Without transparency, you can't possibly catch them. But what might emerge over time as our verification processes improve, as we have a sense of accountability to our transparency,
then the fact that transparency is there may decrease error over time, but not the need to check. And that's the key.
This is a real challenge that we wrestle with and have wrestled with since the origins of the center is how do we promote this culture of critique and self-criticism about our field and and simultaneously have that be understood as the strength of research rather than its weakness.
One of the phrases that I've liked to use in this is that the reason to trust science is because it doesn't trust itself. That part of what makes science great as a social system is its constant self-scrutiny and willingness to try to find and expose its errors So that the evidence that comes out at the end is the most robust, reliable, valid evidence as could be.
And that continuous process is the best process in the world that we've ever invented for knowledge production. We can do better. I think our mistake in some prior efforts of promoting science is to appeal to authority, saying you should trust science because scientists know what they're doing. I don't think that's the way to gain trust in science because anyone can make that claim.
Appeals to authority are very weak arguments. I think our opportunity as a field to address the skepticism of institutions generally and science specifically is is to show our work, is by being transparent, by allowing the criticism to occur, by in fact encouraging and promoting critical engagement with our evidence.
That is the playing field I'd much rather be on with people who are the so-called enemies of science than in competing appeals to authority. Because if they need to wrestle with the evidence and an observer says, wow, one group is totally avoiding the evidence and the other group is actually showing their work, I think people will know who to trust. That's easy to say. It's very hard to do.
These are hard problems.
We are a carrot-based organization because we don't have sticks. I mean, would you like me to loan you a stick just once in a while? Yeah, that would be fun.
Showing 21–40 of 65 · page 2 of 4
← Previous
Next →