Zico Colter
speaker
161 appearances
1 recordings
1 series
first heard Sep 2024
last heard Sep 2024
Zico Colter’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
The way we think about fixing this normally is, you know, we would have certain models, models that we release. We would say, you know, don't use your model ability that you have inside of you to sort of create obvious cyber attacks against certain infrastructure and stuff like this, right? Don't do that. But we can't make them follow that instruction, right?
Someone either with access to a model itself, certainly, but even with access sometimes to a closed source model just can have the ability to jailbreak these things and oftentimes get access to these things, right? To be very clear, we are making immense progress in solving this problem of sort of preventing jailbreaks, kind of avoiding making sure models follow a spec.
But until we can solve this problem, it's very hard to say, you know, all the other dangerous capabilities that AI could sort of demonstrate become much, much more concerning. And so this is kind of a multiplier effect on everything else bad these models can do, which is why I'm so concerned about it right now.
So that's sort of a good lead in, right? Because if jailbreaks and sort of manipulation of models is the attack vector, what is the payoff? What are the things we can do? And here, what we're trying to do really is we're trying to assess the core harmful capabilities of models, right? And people have thought a lot about this, right?
People think about things like creating chemical weapons, creating biological weapons, creating cyber attacks. Personally, I think cyber attacks are a much more clear and present threat than, for example, bio threats and things like this. At the same time, I don't want to dismiss any of these concerns, right?
I think people have looked at this much, much more than myself and are very concerned about these things. So I want to treat this with the respect that honestly it deserves because these are sort of massive problems. There are a lot of potential harms of AI models. Some are associated primarily with scale and things like this, like the misinformation you mentioned.
But some are just, there are capabilities that we think these models might enable where they would lower the bar so much for some bad things, like, say, creating a zero-day exploit that takes down software over half the world. The concern is that, not that they can do this sort of autonomously, maybe initially, but
but that they can lower the bar so far in the skill level required to create these things that effectively it puts them in the hands of a huge number of bad actors. And the same is true for things like biological risk or chemical risk or other things like this. And these concerns have to be taken seriously.
And they have to be things that we really do consider as genuine possibilities if we start putting into everyone's hand the ability to create
Two issues there. One is AI as dangerous as nuclear weapons, and what does this imply about the open release of certain models? So I'll make two points on this. I think the nuclear weapon analogy is actually not a great one, because nuclear weapons have one purpose, which is to destroy things.
Maybe a better analogy is sort of nuclear technology period, because it has the ability to create nuclear weapons, but it also has the ability to do things like provide power, non-CO2 emitting power to potentially a huge number of people, right? A lot of people are currently making a bet on nuclear as the way we create carbon-free energy.
But I think the analogy of nuclear weapons in particular is often overstated precisely because AI has many good uses. Nuclear weapons, arguably, they do one thing, and it's not considered a good use, right? So there's a very different kind of technology there.
But let me get to your second point now, which is the sort of the open model debate, which is also one that frequently is played out in kind of discussions on AI safety. I should start off by saying I'm a fan of open source models in a general sense.
So I want to start by saying that because honestly speaking, open source release of models and I really say open weight because oftentimes these are not actually open source traditional way. They're actually much more like closed source executables. They just you can run them on your own on your own computer. Open weight models. have advanced my ability to study these systems.
They've been the primary tool by which we conduct research in academia and beyond, and they are becoming, I would argue, a critical part of the overall ecosystem of AI right now, number one. Number two,
If you look at the current best models that there are right now, so things like GPT-4, Claude 3.5, Gemini, things like this, I would not currently be all that nervous about having an open source model that was as capable of these in terms of the catastrophic effects of it. Because these models actually aren't by themselves. We have a good handle on them, right?
We sort of know what they're capable of. Arguably, we're already here because Lama 3, 405 billion is pretty close. I don't think it's quite at that level yet, but it's getting there. And, you know, this release has not yet caused some catastrophic event. Because the reality is these models, they still have a ways to go.
Right now, to a certain extent, I think things are okay with open weight release of the models. However, there will come a time when a certain capability, a certain ability of these models reaches the point that should give us pause when it comes to just turning these things over to whoever and everything. what, however they want to use them.
And I do think this, there are certain levels of capabilities that you could see that are within kind of eyesight of our current development. That if I was sort of just to ask the question, no, should, should we give this to everyone, not just to use, but to use and tune and specialize however they want.
And I would just sort of say, I think there will be a point where I get uncomfortable with that. What is that point? So if you think about a model that really could analyze any code base or even any binary executable or website or JavaScript or anything like this and immediately find a vulnerability that it could exploit to take down a large portion of the internet or a large portion of software.
Showing 101–120 of 161 · page 6 of 9
← Previous
Next →