Gary Marcus

But the thought, the scent of money, possibly misguided scent of money, really changed how the field grew.

And it's also, you know, a technical thing.

Transformers are interesting and people got into them.

But fundamentally, I think it's the sense of money really changed how people built AI, how they thought about it, what they wanted to do with it, who was running it.

I think a lot of grifters came in that don't even necessarily have a technical understanding of the questions and do a lot of lying and hyping about what their things might actually do.

And it's just really been unpleasant for the last several years being honest about it.

And it's not because I don't want AI to succeed.

I still think that there's a chance that AI could help a lot in medicine, that it could help with all kinds of technologies.

Like, I would still like to see AI succeed, but not on the path that we are right now.

This is just not a good path.

The technical problem is that large language models, fewer large language models, are basically next token predictors.

That's what they do.

That is literally how they are built, is to predict predictability.

in a sequence of words or other kinds of tokens what might come next and that's an interesting thing to do it's part of what humans do is we do some prediction but it's not all of what cognition is right cognition right intelligence is about cognition about understanding things and so forth

has many different components to it.

And they're just not really built into LLMs.

And so LLMs basically fake everything else.

And we can talk about some complications, people are building in harnesses, and we can go there.

But let's just talk about pure LLMs.

What they do is predict the next token.