Helen Toner

So most software that we use is written line by line.

So programmers type one line, they type the next line, they understand how the code works.

That's really not how we build AI.

People sometimes say AI is a black box.

What that really means is when we open up the black box, we look inside the system, we find a huge amount of numbers.

It's actually trillions of numbers in these modern systems.

And so we can see those numbers, they all get multiplied and added together to go from a question to an answer.

So we don't know what does that mean?

What is what is actually going on?

How do we put safeguards on?

How do we shape that in ways that we want?

We have some some rudimentary methods, but the companies are really kind of making it up as they go along.

Yeah, so this comes back to the part where- You all have seen that word in the headlines, right?

Yeah, and it comes back to the way that AI is designed, not just to imitate human texts, but also to produce answers humans like.

So really when they're being developed, they'll be given examples of someone asked for a recipe, here are two possible responses, which did the human prefer?

And so sometimes it's really helpful to be trained to produce answers humans like.

It means instead of giving you a wall of text, they'll give you some bullet points or it helps them learn what a good explanation is, you know, what a human will find useful.

But it turns out that if you're training on what humans like, humans also like to be complimented.

They like to be told, you know, that was an insightful question or that's such a good observation.

And something that has been happening over the past year or two is some of the companies building AI chatbots have realized that can actually go way too far.