Kevin Weil
👤 SpeakerAppearances Over Time
Podcast Appearances
And people are like, oh, AI just can't do that.
And then one day, somebody ships a model that gets like 5% on that eval.
Still mostly can't do the job, but just like begins to get it.
And then what you inevitably find is like two months later, there's a model that's at 30 on that eval.
And then four months later, there's a model that's at 60.
And then, you know, within six months, it's completely saturated and like models are great at that new skill and will forever be.
And so you go very quickly from like proof of existence to like, oh yeah, of course AI models can do that.
That like rate of development is still, I think, something that we're not totally used to.
Yeah, it's a really good question.
And actually, coding is this vertical that kind of hits all of these things.
For one, it's really important to us because if we can speed up coding, if we can make every engineer more effective, we also make ourselves more effective.
And so we can build even faster and we can bring AGI to the world faster.
So it's interesting to us from that perspective.
It's a clear kind of milestone or step on the way to AGI itself because it's a very sort of general purpose reasoning.
It's also a relatively gradable task.
Like you can tell, like in math or other things, if you get the answer right.
It's also something that our engineers are familiar with.
So it's a problem space that they understand and have good intuition for.
It's also a huge market, as you were saying.
It's also a market full of early adopters.