Anthony Aguirre

if we keep going down the road of building more and more powerful general autonomous intelligences that humanity is basically going to stay in charge of the earth that probability seems to be quite low almost everything we consume as information about the world is being chosen for us by an algorithm that we don't control and don't even understand how it operates and is optimizing something that is not in our best interest necessarily we are currently

sort of seeing a large scale AI that is controlling us rather than being controlled by us as humanity.

Superintelligence is not going to be something that grants you power.

Superintelligence is going to be something that absorbs power.

Yeah, we're a nonprofit that's been around for about 10 years thinking about AI and other transformative technologies and how we can make them go well.

Have them be like a large scale benefit to humanity, not a disaster.

Yeah, we did the pause letter back shortly after GPT-4 came out.

Still think we should have paused.

We didn't.

But we will keep pushing for just doing things that make some sense.

Six months was sort of an amount of time, you know, it was minimum six months, first of all.

But it's sort of minimal amount of time to really get the conversation going and the discussion between the companies and the governments and everyone else to like take a pause and think, well, where are we going with this?

Like, what's the plan?

So we didn't do that.

We're going forward with no plan.

And that's a shame.

But I think it's not too late.

We, I think, still could take a breath.

And rather than sort of racing headlong into more and more powerful models and this race for super intelligence, we could still pause.

But I think the, you know, the way we would put it now is not so much pause as like, let's change direction.