Eliezer Yudkowsky

We have to make sure we understand how these systems function so that we can predict their effect on economy so that there's a... So there's a futile moral panic and a bunch of op-eds in the New York Times and nobody actually stepping forth and saying, you know what, instead of a mega yacht, I'd rather put that billion dollars on prizes for young hotshot physicists who make fundamental breakthroughs in interpretability.

I hope, I hope, I guess.

You want to bet me on that?

You want to put a timescale on it?

Say how much funds you think are going to be allocated in a direction that I would consider to be actually useful?

By what time?

Like, yeah, you could take...

The last generation of systems, the stuff that's already in the open, there is so much in there that we don't understand.

There are so many prizes you could do before you would have enough insights that you'd be like, oh, well, we understand how these systems work.

We understand how these things are doing their outputs.

We can read their minds.

Now let's try it with the bigger systems.

Yeah, we're nowhere near that.

There is so much interpretability work to be done on the weaker versions of the systems.

I mean, the trouble is, the stuff is subtle.

I've watched people try to make progress on this and not get places.

Somebody who just gets alarmed and charges in, it's like going nowhere.

It meant years ago, about 20 years, 15 years, something like that, I was talking to a congressperson.

who had become alarmed about the eventual prospects, and he wanted work on building AIs without emotions, because the emotional AIs were the scary ones, you see.

And some poor person at ARPA had come up with a research proposal whereby this congressman's panic and desire to fund this thing would...