Stephen Wolfram

And by the way, the thing to understand is it's putting down one word at a time.

But the outer loop of the fact that it says, okay, I put down a word.

Now let's take the whole thing I wrote so far.

Let's feed that back in.

Let's put down another word.

That outer loop, which seems almost trivial, is really important to the operation of the thing.

And for example, one of the things that is kind of funky is it'll give an answer.

And you say to it, is that answer correct?

And it'll say, no.

And why is that happening?

It's fascinating, right?

Why can it do that?

Well, the answer is because it is going, one word at a time, sort of forwards.

And it didn't, you know, it came along with some sort of chain of thought, in a sense, and it came up with completely the wrong answer.

But as soon as you feed it, the whole thing that it...

came up with, it immediately knows that that isn't right.

It immediately can recognize that was a bad syllogism or something and can see what happened, even though as it was being led down this garden path, so to speak, it came to the wrong place.

That's quite, I'm not sure what to make of it.

Well, look, I think there are many things we don't understand.

Okay, so for example, 175 billion weights, it's maybe about a trillion bytes of information, which is very comparable to the training set that was used.