Warren Hatch

And so you're better off by going to a crowd who are very skilled at assigning probabilities that occur with that frequency to give a forecast for that particular thing. Now, best of all is if you get an expert who applies themselves and becomes a good forecaster, that's what we really want to see. But experts, not necessarily good forecasters, just by being an expert.

And so you're better off by going to a crowd who are very skilled at assigning probabilities that occur with that frequency to give a forecast for that particular thing. Now, best of all is if you get an expert who applies themselves and becomes a good forecaster, that's what we really want to see. But experts, not necessarily good forecasters, just by being an expert.

Oh, I love that. That's a great question. And that's half the work is getting the question right. You want to make sure that everybody understands it in the same way. You want to be sure that it's actionable, it's useful. Why go to all this effort if it's just, you know, a parlor game?

Oh, I love that. That's a great question. And that's half the work is getting the question right. You want to make sure that everybody understands it in the same way. You want to be sure that it's actionable, it's useful. Why go to all this effort if it's just, you know, a parlor game?

And you also want to be able to say, well, when that date occurs and you look back, we will agree it happened or it didn't. And a lot of forecast questions that are out there don't meet those criteria. There was a great example because in the original part of the project, the government wrote the questions themselves.

And you also want to be able to say, well, when that date occurs and you look back, we will agree it happened or it didn't. And a lot of forecast questions that are out there don't meet those criteria. There was a great example because in the original part of the project, the government wrote the questions themselves.

And in that first year or so, they had questions of the sort like, will construction begin on a canal through Nicaragua? And this was back when there was a guy in Hong Kong who was going to build a big, giant canal for the Supermax boats. And will construction begin there? was the question that they posed. Now, if you reflect a little bit, you can see that there's a problem with that.

And in that first year or so, they had questions of the sort like, will construction begin on a canal through Nicaragua? And this was back when there was a guy in Hong Kong who was going to build a big, giant canal for the Supermax boats. And will construction begin there? was the question that they posed. Now, if you reflect a little bit, you can see that there's a problem with that.

What counts as construction? Is it built with boats going through? Or is it big roads going through the jungle? Or is it a golden shovel in the ground? Which was my view. And sure enough, there was a golden shovel in the ground. But that was it. So did construction begin? We need to agree on what the question's asking. So the head of our question team

What counts as construction? Is it built with boats going through? Or is it big roads going through the jungle? Or is it a golden shovel in the ground? Which was my view. And sure enough, there was a golden shovel in the ground. But that was it. So did construction begin? We need to agree on what the question's asking. So the head of our question team

The fail rate we have in our questions now is like 0% this year, thanks to his hard work. And what he'll do is in addition to writing a careful question, he'll include, it's almost like a little contract. He's trained as an attorney where this is how the question will resolve under these circuits. And for things that matter, it's worth going to that effort.

The fail rate we have in our questions now is like 0% this year, thanks to his hard work. And what he'll do is in addition to writing a careful question, he'll include, it's almost like a little contract. He's trained as an attorney where this is how the question will resolve under these circuits. And for things that matter, it's worth going to that effort.

Not everything matters that much, but for things that do, then you want to make sure it's crisp and tight. It's actionable, it's verifiable, and we all can agree if it happened or it did not.

Not everything matters that much, but for things that do, then you want to make sure it's crisp and tight. It's actionable, it's verifiable, and we all can agree if it happened or it did not.

For the higher order questions, things that really impact our lives, it's often not just a single decision. There might be a series of smaller decisions that go into it.

For the higher order questions, things that really impact our lives, it's often not just a single decision. There might be a series of smaller decisions that go into it.

and so breaking that down into what are the pieces that are really going to be impactful for how i think about this issue so where to go to school that's more than just a single decision there's a series of them do they provide the support that i would want as a student do they have the right kind of curriculum what sorts of career trajectory might i get on those are all smaller sub decisions and those are all things that you can then analyze as a forecasting question too

and so breaking that down into what are the pieces that are really going to be impactful for how i think about this issue so where to go to school that's more than just a single decision there's a series of them do they provide the support that i would want as a student do they have the right kind of curriculum what sorts of career trajectory might i get on those are all smaller sub decisions and those are all things that you can then analyze as a forecasting question too

Now, one thing that people tend to do, both when they're thinking about the question as well as the forecast, is focus on the particulars themselves. Like who's going to win the next presidential election? Most people immediately start thinking about the candidates. And what we want to do is instead of going to the particulars right away, we want to zoom out. How do things like this usually go?

Now, one thing that people tend to do, both when they're thinking about the question as well as the forecast, is focus on the particulars themselves. Like who's going to win the next presidential election? Most people immediately start thinking about the candidates. And what we want to do is instead of going to the particulars right away, we want to zoom out. How do things like this usually go?