Stephen Wolfram

Yeah, so what does something like ChatGPT do?

It's mostly focused on make language like the language that humans have made and put on the web and so on.

So its primary sort of underlying technical thing is you've given a prompt, it's trying to continue that prompt in a way that's somehow typical of what it's seen based on a trillion words of text that humans have written on the web.

The way it's doing that is with something which is probably quite similar to the way we humans do the first stages of that, using a neural net and so on, and just saying, given this piece of text, let's ripple through the neural net and get one word at a time of output.

And it's kind of a shallow computation on a large amount of kind of training data that is what we humans have put on the web.

That's a different thing from sort of the computational stack that I spent the last, I don't know, 40 years or so building, which has to do with what can you compute many steps, potentially a very deep computation,

It's not taking the statistics of what we humans have produced and trying to continue things based on that statistics.

Instead, it's trying to take the formal structure that we've created in our civilization, whether it's from mathematics or whether it's from systematic knowledge of all kinds,

and use that to do arbitrarily deep computations to figure out things that aren't just let's match what's already been kind of said on the web, but let's potentially be able to compute something new and different that's never been computed before.

So as a practical matter,

You know, our goal is to have made as much as possible of the world computable in the sense that if there's a question that in principle is answerable from some sort of expert knowledge that's been accumulated, we can compute the answer to that question and we can do it in a sort of reliable way that's the best one can do given what the expertise that our civilization has accumulated.

It's a much more labor-intensive on the side of creating the computational system to do that.

Obviously, in the chat GPT world, it's like, take things which were produced for quite other purposes, namely all the things we've written out on the web and so on,

And sort of forage from that things which are like what's been written on the web.

So I think as a practical point of view, I view sort of the chat GPT thing as being wide and shallow.

And what we're trying to do with sort of building out computation as being this sort of deep thing.

Also broad, but most importantly, kind of deep type of thing.

I think another way to think about this is if you go back in human history, you know, I don't know, a thousand years or something, and you say, what's the typical person going to figure out?

Well, the answer is there are certain kinds of things that we humans can quickly figure out.

That's sort of what our neural architecture and the kinds of things we learn in our lives let us do.