Azeem Azhar

It's worth saying that, you know, the evidence for actual value is, in my view, stacked toward there being value, but there is contrary evidence emerging.

That MIT paper, which made it into the Financial Times and elsewhere and amongst investors, is worth noting.

exploring.

I wrote about it in an issue of the newsletter a couple of weeks ago.

It said essentially that enterprises said that in 90 or 95% of the case, generative AI projects were not working.

But I had

some real kind of issues and problems with the methodology of that paper in terms of the way in which participants were sampled, the way in which the data was gathered.

It was not a representative sample.

It didn't feel like the questioning adhered to the kind of standard that you'd expect

for an academic paper.

And also I felt the timeframe was a bit awry.

IT projects, any kind of project in a company takes a few years, a couple of years at least to start to pay back.

And even with the best will of the world, because we misspecify these projects,

A large portion of them, 60-70% of them, will fail.

And to ask in six months whether these things are working, well, it just feels like it's way too short of a time period.

It's a bit like getting angry with the pregnant cow elephant because four months into her 15-month gestation period, she hasn't given birth.

I mean, it's just too short a period.

So yes, I read the paper.

I thought, obviously, it was of interest to people in the world, but I don't think it necessarily pours cold water on it.

But I think this is a really important point, which is we don't really have super robust evidence about this.