Gwern Branwen

There, they're actually training agents.

Instead, we train them on these internet scrapes

which merely encode the outputs of agents or occasional descriptions of agents doing things, that kind of thing.

There's no actual like logging of state environments, result reward trip sequences, like a proper kind of reinforcement learning setup would have.

I would say that what's more interesting actually is that nobody wants to train agents in a proper reinforcement learning way today.

Instead, everyone wants to train LLMs and then do everything with as little RL as possible on the backend.

I think if the internet didn't exist, I would have...

tried to probably make it in regular academia and maybe narrow my interests a lot more, something I could publish on regularly.

Or I could possibly have tried to opt out, you know, and become a librarian, like one of my favorite writers, Jorge Luis Borges.

He was a librarian until he succeeded as a writer.

Of course, I've always agreed with him about imagining Paradise as a kind of library.

I regret that all the reading I do is now kind of on the computer and I don't get to spend as much time in libraries, physical libraries.

I genuinely love them, just like pouring through the stacks, looking for random stuff.

Some of the best times for me when I was in university were always like going through these gigantic stacks of all sorts of obscure books and just looking at like a random spine, you know, pulling stuff off the shelf and reading obscure old technical journals to see all the strange and wonderful things that they were doing and documenting back then, which now have just been totally forgotten.

He's a real hero of mine, so this isn't something I want to have a bad answer to.

Can I ask why he's a hero of yours?

When I was younger, one of the science fiction books that really impressed me was Dan Simmons' Hyperion, and especially the fall of Hyperion.

In there, he alludes to Kevin Kelly's Out of Control book, which strongly features the parable of the Library of Babel.

From there, I got the kind of collected editions of Borges' fiction and nonfiction, and I just read through them again and again.

I was blown away by the fact that you could be so creative with all of this polymathic knowledge that he had in erudition and write these wonderful, entertaining, provocative short stories and essays.