Gary Marcus

And if you train them on the entire internet, which is what people in fact do, they can make a pretty good approximation of human beings and how they talk and so forth.

But that approximation is very superficial.

It's very data dependent.

And when you push them outside of the regime in which they've been trained, they will do really stupid things.

So, like...

A couple years ago, there were all these examples of so-called river-crossing problems, like you have a man and a goat and a woman, and they have to go across.

And these systems would say the most absurd things in response to those problems.

It got so embarrassing that Anthropic built in river-crossing problems into their system prompts to try to keep the systems from making these embarrassing errors.

And what the embarrassing errors revealed

is the systems are not really reasoning about things like a man or a river or a boat or what it means to go across the other side.

They're just trying to kind of glom the words together that they have seen.

I mean, you know, the technical details are a little bit complicated, but to a first approximation, what they are doing

is just stringing these words together.

There are other ways to think about building intelligence.

So you might start, for example, with a database, who did what to whom, when and where.

If you actually did that, if you started with that, you would not have all these crazy hallucinations.

And so here we are, you know, in 2026, I started writing about LLMs in 2019.

And I said, they don't have stable models of the world.

You can't trust on them.

And everybody said, Gary, Gary, Gary, we're just going to add more data.