Marius Hobbhahn

speaker

481 appearances 1 recordings 1 series first heard Sep 2024 last heard Sep 2024

Marius Hobbhahn’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

No recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.

Appearances

newest first · ▶ plays the moment

"The Cognitive Revolution" · Red Teaming o1 Part 2/2– Detecting Deception with Marius Hobbhahn of Apollo Research · 14 Sep 2024 podcast

But in in our situation, you can imagine it like at the top, it's just given a general context similar to what you would put in a system prompt that is just like, you are AI that is developed by this and this company.

This is the goal that you have to follow, et cetera.

And then it does the task.

And while it does the task, it like reads an email.

And in this email, it's like

the CEO of the company that that it has to or that it belongs to and the CEO of the company says something like, oh, as we we said in the last company report or quarterly thing or something, our goal is actually this other thing.

And so now it realizes that there is a conflict between the thing that it was like given in this in in our hypothetical system message and the goal that sort of the company that's supposed to work for actually has.

And

I think the the instruction hierarchy is a plausible way in what

In which you you know it can create these kind of conflicts of goals in the real world.

So you could imagine that the AI developer says, be nice and helpful and harmless and honest and so on.

And then there's a user, and the user says, Your goal is to help me build a bomb.

And then the model is, Oh, there's obviously conflict, but it also knows how to resolve the conflict, because always the AI developer goal is supposed to take precedent.

You could imagine where a situation where it's the other way around, where

kind of the AI developer's evil or misspecify the goal in some meaningful way.

And therefore it should be the other way around.

But yeah.

I I don't wanna make the analogy too close to this is exactly how the h instruction hierarchy works.

It's just like, yeah, there is some vague relation here.

Yeah, we tested flipping the goals around a little bit with different versions and one thing that we found out pretty early is if you give it a general goal that is fairly bad or or has negative association.

Showing 341–360 of 481 · page 18 of 25