Rob Wiblin

I guess one thing that I was glad that you said that not everyone would highlight is you need to treat an AGI or an AI inside the company that's just only being deployed internally.

If you're not sure or not confident of its alignment, then you need to treat it as like an untrusted insider, as potentially a hostile agent that is going to try to undermine internal systems.

Yeah, do you want to say anything about, like highlight any of the things that you are committing to doing?

Sorry, yeah.

Planning.

Let's talk about timelines and recursive self-improvement loops for a bit.

There's been a lot of discussion recently about when we might expect a recursive self-improvement loop to occur, indeed, if one is possible at all.

Do you think that people have been a little bit sloppy in their thinking about this in some ways and in ways that cause people to maybe expect it to happen sooner than you do?

Explain that.

Okay, let me recap all of that.

So we currently have exponential economic growth, which is to say a constant percentage growth each year on average, looking over the medium term.

People who say there's going to be an intelligence explosion, there's going to be a recursive self-improvement loop, they're forecasting increasing rates of growth or hyperbolic economic growth.

So it's 3% one year, then 10% the next, then 50% the next, up to some point, I guess, at which you might think it levels off or comes back down again.

Looking, what are the high level factors that lead economic growth to either be stable or to increase or to go back down?

There's three factors that you have in mind in your model and that most economists have in their model.

As technology advances, it gets difficult to make new useful discoveries and to advance it further.

As technology advances, that's the first one.

The second one, as technology advances, we have better tools to do science and to make new discoveries.

The third one is, as technology advances and the economy grows, we can support a larger number of people who will do the research and have the ideas and advance science.

In recent times, the third one has kind of been out of the picture because we haven't been turning advances in science or improvements in the economy into new people.