Mike Knoop

I went back to be an individual contributor and just to go do AI research alongside Brian, my co-founder.

Um, and ultimately that led me to back towards arc.

Uh, I was looking into it again and I had sort of expected to see, uh, this, you know, saturation effect that, you know, MMLU has that GMSK has eight K has.

And when I looked at the scores and the progress that since the last four years, um, I was really, again, shocked to see, actually we've made very little objective progress towards it.

And, um, it felt very, it felt like a really, really important eval.

And as I sort of spent the last year asking people, quizzing people about it and sort of my networking community, um,

Very few people even knew it existed.

And that felt like, okay, if it's right that this is a really, really globally singularly unique AGI eval, and it's different from every other eval that exists that more narrowly measures AI skill,

Like more people should know about this thing.

I had my own ideas on how to beat the arc as well.

So I'm like, I was working on nights and weekends on that.

And I flew up to meet Francois earlier this year to sort of quiz him, show him my ideas and ultimately led.

I was like, well, you know, why don't you think more people know about arc?

I think you should actually answer that.

I think it's a really interesting question.

Like why don't you think more people know about arc?

MARK MANDELMANN- This is what nerds night me, because I went and took a bunch of the puzzles myself.

I've showed it to all my friends and family, too.

And they're all like, oh, yeah, this is super easy.

are you sure AI can't solve this?