Dwarkesh Patel

So is it surprising that they aren't able to integrate that into whenever you're like add rope embeddings or something, they do that in the wrong way?

Yeah.

Actually, here's another reason why this is really interesting.

Through the history of programming, there's been many productivity improvements, compilers, linting, better programming languages, etc.,

which have increased programmer productivity, but have not led to an explosion.

So that sounds very much like autocomplete tab.

And this other category is just like automation of the programmer.

And so it's interesting you're seeing more in the category of the historical analogies of better compilers or something.

One of the big problems with RL is that it's incredibly information sparse.

LabelBox can help you with this by increasing the amount of information that your agent gets to learn from with every single episode.

For example, one of their customers wanted to train a coding agent.

So LabelBox augmented an IDE with a bunch of extra data collection tools and staffed a team of expert software engineers from their aligner network to generate trajectories that were optimized for training.

Now, obviously these engineers evaluated these interactions on a pass-fail basis, but they also rated every single response on a bunch of different dimensions like readability and performance.

And they wrote down their thought processes for every single rating that they gave.

So you're basically showing every single step an engineer takes and every single thought that they have while they're doing their job.

And this is just something you could never get from usage data alone.

And so LabelBox packaged up all these evaluations and included all the agent trajectories and the corrective human edits for the customer to train on.

This is just one example, so go check out how Labelbox can get you high-quality frontier data across domains, modalities, and training paradigms.

Reach out at labelbox.com slash thwarkesh.

Let's talk about RL a bit.