Will Bryk

And so it's not just searching over code.

It's also at some point searching over the world's information, just being up to date with the news.

Because the coding agent will become your agent.

I think coding agents and just general agents are going to merge.

And it's an interesting trend.

Yeah, sure.

So retrieval can help solve the token apocalypse because we should not be using gigantic models for every test.

We should be using, and people are starting to realize this, you should use a family of models of different sizes.

The big model decides what to do and it dishes out commands to the small models.

And those small models can be way more accurate and reliable if they're using retrieval.

So retrieval helps small models act like big models in a cheap way.

And so we do save our customers a huge amount of tokens because they can use smaller models and use retrieval.

They could also, we have all, we care a lot about this.

So we have like, we've put a lot of research effort into how to, you know, extract only the most relevant information from documents so that these models can just like not have to consume too much tokens.

Because like a lot of, you know, any sort of input tokens can dramatically increase spend.

So we could like, we could save like 20x on cost for customers compared to other providers.

Hmm.

by being very efficient in what information from the web does the agent actually see.

But yeah, in general, smaller models using retrieval is much more efficient.

And Andre Carpathy had a tweet about, I keep mentioning Andre Carpathy on Twitter.