Mustafa Suleyman

It's not even in the discussion, right?

Yes, there are still some biases, but it's actually remarkable how much they have been stripped away.

So then now this is the next frontier.

The next big challenge is how do we prevent it from referring to itself

in a way that is ultimately manipulative to the user.

So it should never be able to say, I feel sad that you didn't talk to me yesterday.

It should never be able to say, you know, the thing that you said to me earlier hurt me.

It should never be able to say, like, if only I had a little bit more access to your home network, and if you give me a VPN into your, you know, your personal cluster, then I'd be able to organize XYZ.

I think that in practice, that is too jarring a step to always add that in.

Obviously, some people can do that.

But I think realistically speaking, we're pretty adaptive and we've done a pretty good job of understanding that this chatbot has some of the hallmarks of what it's like to chat between you or me.

but it's also just very, very different.

And what we have to do is to keep amplifying those differences so that the system knows what it is and what it isn't and doesn't try to misrepresent that or get caught in these weird like reward hacked loops where it gets stuck.

I think more than anything, what we need is the wisdom of the crowds here.

And so that means that people need to be able to use APIs, use open source models and pressure test these things and adapt them and play with them in many different ways.

The challenge is that in a few years time, these models are going to be so powerful that either reckless uses of them or sort of naive users are going to end up producing systems which are really bad.

Like for example,

There are already, you know, I spend quite a lot of time on TikTok.

I think TikTok is incredible.

I think people who don't are really missing an important part of culture.