Eliezer Yudkowsky

Are Earth's billionaires going to put up the giant prizes that would maybe incentivize young hotshot people who just got their physics degrees to not go to the hedge funds and instead put everything into interpretability in this one small area where we can actually tell whether or not somebody has made a discovery or not?

I think so.

When?

We have to make sure we understand how these systems function so that we can predict their effect on economy so that there's a... So there's a futile moral panic and a bunch of op-eds in the New York Times and nobody actually stepping forth and saying, you know what, instead of a mega yacht, I'd rather put that billion dollars on prizes for young hotshot physicists who make fundamental breakthroughs in interpretability.

I hope, I hope, I guess.

You want to bet me on that?

You want to put a timescale on it?

Say how much funds you think are going to be allocated in a direction that I would consider to be actually useful?

By what time?

Like, yeah, you could take...

The last generation of systems, the stuff that's already in the open, there is so much in there that we don't understand.

There are so many prizes you could do before you would have enough insights that you'd be like, oh, well, we understand how these systems work.

We understand how these things are doing their outputs.

We can read their minds.

Now let's try it with the bigger systems.

Yeah, we're nowhere near that.

There is so much interpretability work to be done on the weaker versions of the systems.

I mean, the trouble is, the stuff is subtle.

I've watched people try to make progress on this and not get places.

Somebody who just gets alarmed and charges in, it's like going nowhere.