Peter McCrory

If you don't have the relevant information to deploy that capability, business adoption will maybe be constrained or bottlenecked.

Exactly.

The tacit knowledge piece is so crucial, I think, about developing a sales strategy.

It might be that your co-worker has the relevant information, not even in the

customer management software that you're using, but in their brain or in some document that they've written out.

And if the business is not thinking about how to elicit that tacit information in a way that can make the model effective, you might fail to see tasks in our API deployment for which Claude and other frontier models would actually be capable if they had that information.

Yeah, the data, the underlying data that we sampled was just before Opus 4.5.

This sort of touches on one of the findings, which is for the most complex tasks, which I presume you're asking Opus 4.5 to take on very ambitious exercises.

In our data, those most complex tasks tend to have, that's where the model struggles.

So there's like the success rate is somewhat lower.

And that suggests that human expertise to evaluate the quality of the output

is more important and you need more human delegation and direction and managerial oversight.

I have found that even with Opus 4.5, there's still some level of oversight and quality attention that I need to maintain, but I actually worry a lot less about the implementation steps.

And so I'll give somewhat of a concrete example.

where analysis that I've been wanting to do for a while, and I haven't had the time to do it.

And it sort of touches on this idea of how does AI exposure across jobs as commonly constructed?

How does that relate to business cycle sensitivity?

Is it the case that workers who have high AI exposure, are they the ones who

might have higher unemployment when the labor market slows down.

I've been giving Claude for some time now in a sort of separate server to run autonomously this exercise of downloading some research papers, implementing the sort of replicating some prior reports, and then coming up with this like new analysis that I wanted to do.