Noam Shazeer

and steer it as much as we can away, maybe with policy-related things, maybe with technical measures and safeguards, away from the computer will take over and have unlimited control of what it can do.

So I think that's an engineering problem is how do you engineer safe systems?

I think it's kind of the modern equivalent of what we've done in kind of older style software development.

Like if you look at, you know, airplane software development, that has a pretty good record of how do you rigorously develop safe and secure systems for doing a pretty risky job.

risky task.

Where did we get the training data?

I mean, I think having the system explore algorithmic research ideas seems like something where there's still a human in charge of that.

It's exploring space and then it's going to get a bunch of results and we're going to make a decision.

Are we going to incorporate this particular learning algorithm or change the system?

Yeah.

into kind of the core code base.

And so I think you can put in safeguards like that that enable us to get the benefits of the system that can sort of improve or kind of self-improve with human oversight without necessarily letting the system go full-on self-improving without any notion of a person looking at what it's doing, right?

That's the kind of engineering safeguards I'm talking about where you want...

to be kind of looking at the characteristics of the systems you're deploying, not deploy ones that are harmful by some measures and some ways you have in understanding what its capabilities are and what it's likely to do in certain scenarios.

So, you know, I think it's not an easy problem by any means, but I do think it is possible to make these systems safe.

One thing I would say is if you expose the model's capabilities through an API or through a user interface that people interact with,

You know, I think then you have a level of control to understand how is it being used and sort of put some boundaries on what it can do.

And that, I think, is one of the tools in the arsenal of like how do you make sure that what it's going to do is sort of acceptable by some set of standards you've set out in your mind.

I mean, I think the early sort of four or five years at Google, when I was sort of one of a handful of people working on search and crawling in search and indexing systems, and our traffic was growing tremendously fast, and we were trying to expand our index size and make it so we updated it every minute instead of every month or two months if something went wrong.

And seeing kind of the growth and usage of our systems was really just personally satisfying.