Stephen Wolfram

Yeah.

Yeah, that's a good existential issue.

But then when you say survival of the species, right, you might say, you might, for example, for example, let's say,

Forget about technology, just hang out and be happy, live our lives, go on to the next generation, go through many, many generations where, in a sense, nothing is happening.

Is that okay?

Is that not okay?

Hard to know.

In terms of, you know, the attempt to do elaborate things and the attempt to might be counterproductive for the survival of the species.

Like, for instance, I mean, you know, I think it's also a little bit hard to know.

So, okay, let's take that as a sort of thought experiment, okay?

You know, you can say, well...

what are the threats that we might have to survive?

The super volcano, the asteroid impact, all these kinds of things.

Okay, so now we inventory these possible threats and we say, let's make our species as robust as possible relative to all these threats.

I think in the end, it's sort of an unknowable thing what it takes to... So given that you've got this AI and you've told it, maximize the long-term.

What does long-term mean?

Does long-term mean until the sun burns out?

That's not going to work.

Does long-term mean next thousand years?

They're probably optimizations for the next thousand years.