Noam Brown
speaker
1,199 appearances
1 recordings
1 series
first heard Dec 2022
last heard Dec 2022
Noam Brown’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
And so that is something that seems almost impossible to model purely from scratch without any human data.
It's a very cultural thing.
And so you need human data to be able to understand that, hey, that's how humans behave.
And you have to work around that.
It might be suboptimal, it might be irrational, but that's an aspect of humanity that you have to deal with.
That's actually one of the major challenges that we faced in the research, that we had a good amount of human data.
We had about 50,000 games.
What we try to do is leverage as much self-play as possible while still leveraging the human data.
So what we do is we do self-play, very similar to how it's been done in poker and Go, but we try to regularize the self-play towards the human data.
Basically, the way to think about it is
So we penalize the bot for choosing actions that are very unlikely under the human data set.
Yeah, so we train a bot through supervised learning to model the human play as much as possible.
So we basically train a neural net on those 50,000 games, and that gives us a policy that resembles to some extent how humans actually play the game.
Now, this isn't a perfect model of human play because we don't have unlimited data.
We don't have unlimited neural net capacity, but it gives us some approximation.
Yeah, absolutely.
And so for the language model, which is kind of like a separate question, you know, we didn't use the language model during self-play training, but we pre-trained the language model on, you know, tons of internet data as much as possible.
And then we fine-tuned it specifically on the diplomacy games.
So we are able to like leverage the wider data set in order to fill in some of the gaps in like how communication happens more broadly besides just like specifically in these diplomacy games.
Well, I think there's a few insights.
Showing 981–1000 of 1,199 · page 50 of 60
← Previous
Next →