Warren Hatch
speaker
48 appearances
1 recordings
1 series
first heard Dec 2024
last heard Dec 2024
Warren Hatch’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
Thank you, Lynn. It's a pleasure to be here. Thanks for having me on.
In a sense, every decision that we make is a forecast because we're going to be taking action and doing things to improve the odds that we're going to get our desired outcome, whatever that might be. So we're doing that all day, all week, all year long, or thousands of decisions that way. So we want to make the best possible forecast.
And super forecasting is a process to get to the best possible forecast and therefore to get to the best possible decision.
Well, most people, and it's the dominant way, is they'll use language to express their views about the future. Somebody will ask them, well, do you think this will happen? And they'll say, well, maybe, maybe it will. Or they'll use fancier words like, well, there's a possibility or a distinct possibility. Now, here's the problem with that, many problems.
One problem is we're all going to understand that in different ways. There is a famous example when Kennedy came into office and inherited a plan to invade Cuba and topple the regime. He asked his advisors, will this succeed? And they said, there's a fair chance it will succeed. Now, it turns out Kennedy had in mind north of 50%. The analysts had in mind something more like 25%.
By using language, there was noise in that decision process. I imagine if Kennedy had known they had in mind 25%, history might have been a little different. So that is true for all kinds of words like that, where we're going to interpret it differently. Another bad thing about it is that it straddles for the 50-50 line. So if it happens, you say, aha, I said there'd be a distinct possibility.
If it doesn't, well, I only said it was a distinct possibility. One other bad thing about it is it's impossible to put different views together. You can't crowdsource language like that because we're all using different words and in different ways. How much better to use a number? We all know what 72% is. We all know what that means.
That's a great question, because sometimes there's an apparent tension between experts and good forecasters. where it's one is better than the other. We really are of the view that you want both. You want hybrid models when it comes to experts and things like that. And one core reason for that is experts might be good forecasters. But we don't know that until we see their track record.
Just being an expert does not make you a good forecaster. We want to see if when you say an 80% probability of something occurring, it occurs eight times out of 10 and doesn't two times out of 10. That's accuracy in a probabilistic sense. Most experts do not go through that process.
They're very good at telling us what we need to understand, how we got where we are, some of the things we might want to watch. going forward. But what we've seen time and again, when you ask them for a probabilistic forecast, experts generally tend to assign higher probabilities to events in their area of expertise than actually occur.
And so you're better off by going to a crowd who are very skilled at assigning probabilities that occur with that frequency to give a forecast for that particular thing. Now, best of all is if you get an expert who applies themselves and becomes a good forecaster, that's what we really want to see. But experts, not necessarily good forecasters, just by being an expert.
Oh, I love that. That's a great question. And that's half the work is getting the question right. You want to make sure that everybody understands it in the same way. You want to be sure that it's actionable, it's useful. Why go to all this effort if it's just, you know, a parlor game?
And you also want to be able to say, well, when that date occurs and you look back, we will agree it happened or it didn't. And a lot of forecast questions that are out there don't meet those criteria. There was a great example because in the original part of the project, the government wrote the questions themselves.
And in that first year or so, they had questions of the sort like, will construction begin on a canal through Nicaragua? And this was back when there was a guy in Hong Kong who was going to build a big, giant canal for the Supermax boats. And will construction begin there? was the question that they posed. Now, if you reflect a little bit, you can see that there's a problem with that.
What counts as construction? Is it built with boats going through? Or is it big roads going through the jungle? Or is it a golden shovel in the ground? Which was my view. And sure enough, there was a golden shovel in the ground. But that was it. So did construction begin? We need to agree on what the question's asking. So the head of our question team
The fail rate we have in our questions now is like 0% this year, thanks to his hard work. And what he'll do is in addition to writing a careful question, he'll include, it's almost like a little contract. He's trained as an attorney where this is how the question will resolve under these circuits. And for things that matter, it's worth going to that effort.
Not everything matters that much, but for things that do, then you want to make sure it's crisp and tight. It's actionable, it's verifiable, and we all can agree if it happened or it did not.
For the higher order questions, things that really impact our lives, it's often not just a single decision. There might be a series of smaller decisions that go into it.
and so breaking that down into what are the pieces that are really going to be impactful for how i think about this issue so where to go to school that's more than just a single decision there's a series of them do they provide the support that i would want as a student do they have the right kind of curriculum what sorts of career trajectory might i get on those are all smaller sub decisions and those are all things that you can then analyze as a forecasting question too
Now, one thing that people tend to do, both when they're thinking about the question as well as the forecast, is focus on the particulars themselves. Like who's going to win the next presidential election? Most people immediately start thinking about the candidates. And what we want to do is instead of going to the particulars right away, we want to zoom out. How do things like this usually go?
Showing 1–20 of 48 · page 1 of 3
Next →