Tom Griffiths

You know that you're only going to be able to learn those from your five years of input.

You say, what are initial weights that I can put in my neural network that are going to help that neural network learn each of these languages from that five years of input just using the mechanisms that it has for learning?

The way that we solve that problem using an algorithm that's called model agnostic meta-learning is you have a learning process which has an outer loop and an inner loop.

And the inner loop is just learning an individual language.

So you're just adjusting the weights of the neural network away from those initial weights to learn each of those languages.

But the outer loop is saying, when I look at my performance across all of those languages that I want to learn,

How does my performance on those languages change when I change my initial weights?

And so you can actually learn the initial weights by trying to find initial weights that help to improve performance across all of the languages.

And so by doing that, you're finding a starting point for your neural networks

which is one that's going to allow them to learn quickly from the limited data that they're getting.

And then we can go back and we can look at that set of initial weights and we can say, oh, what does that tell us about the biases that human learners might have?

What are the kind of biases that you need to have in order to be able to learn language from the amount of data that we actually get?

I think it will make them less inclined to find solutions that are not like the human solutions, right?

And that's a plus and a minus.

Yeah, that's right.

Because there are lots of things that humans are really bad at, right?

And we might want to be able to make...

AI systems that can compliment us by being able to do the things that people are bad at.

And that's a really good way of thinking about what a possible future is where humans and AI get to exist side by side in a way which is good for everybody.

I think the thing that might be quite good about making AI systems that have inductive biases that are more aligned with people