Azeem Azhar

Actually, back to 1950, before the term artificial intelligence had been coined.

We'll go to Alan Turing.

He was doing all that breakthrough work in cryptography and the theory of computation.

And he came up with this test for machine intelligence that later on got known as the Turing test.

The test was reasonably simple, right?

If the output of a computer system, a machine, was indistinguishable to other humans from the outputs from other humans, you've got a machine that is exhibiting some type of thinking.

And the Turing test became the thing people measured towards humans.

It persuaded judges at Britain's Royal Institution that it was human years before ChatGPT

And so we end up in this world where we say, well, it used deception.

It's a parlor trick.

This isn't a good test.

Today's LLMs easily pass the Turing test.

And we've already started to see some media outlets having to retract stories that they now realize weren't written by freelancers, but were actually written by people using AI systems end to end.

So today we don't use the Turing test as a test for machine intelligence.

We have shifted the goalposts.

We measure AI's performance against a series of increasingly complex benchmarks.

This effect of shifting goalposts has been noticed since the 1970s.

Rodney Brooks, who is a professor of computer science and robotics at MIT, puts it very, very pithily.

He says,

Every time we figure out a piece of this artificial intelligence, it stops being magical.