Cal Newport

chatting with technology.

You should not be chatting in a sort of anthropomorphized humanized way.

It doesn't mean you can't do natural language processing.

I mean, Google is natural language processing.

You're writing your Google searches in natural language, but no one's having a conversation with Google.

It's you list the keywords as quickly as possible.

And Google's pretty good at figuring out, you know, population, Spain, 1982.

And you press enter and you get that information.

You're not like, hey, so I'm wondering what the population is of Spain in 1982.

Can you help me find that question mark?

There's something odd about that anthropomorphized conversational interface.

I guess we saw a lot of Star Trek growing up.

And that's, you know, what we think the future is supposed to be like.

But it has all sorts of problems.

I think it's hard, actually.

I think it's actually hard to get a language model to do that, right?

Because if you think about...

When you go back to the base layer of what's happening in the pre-training is that you're building a language model that's trying to win at the token guessing game.

So I'm trying to guess what word or part of word actually comes next to what I assume to be a real piece of text.

And then if you do that autoregressively, so you call it again and again and again, adding the answer to the input so it grows out an answer, what you're going to get is text expansion.