David Shu

And, you know, around the same time we had ETags for Emacs or CTags, I should say, which gave us similar things in the Unix world. And so this is sort of extending that idea by bringing out some of the knowledge of a large language model in the process of And I'm really enamored with the entire model. Copilot's original experience when it came out was magical.

And, you know, around the same time we had ETags for Emacs or CTags, I should say, which gave us similar things in the Unix world. And so this is sort of extending that idea by bringing out some of the knowledge of a large language model in the process of And I'm really enamored with the entire model. Copilot's original experience when it came out was magical.

It was just like, there was nothing like this before. It was. And it really, I think, jump-started a lot of interest in the space from people who hadn't been working on it, which was almost all of us. And from my perspective, the thing that really struck me was, wow, this works really well. And wow, it makes really obvious, silly mistakes. There's both sides of this.

It was just like, there was nothing like this before. It was. And it really, I think, jump-started a lot of interest in the space from people who hadn't been working on it, which was almost all of us. And from my perspective, the thing that really struck me was, wow, this works really well. And wow, it makes really obvious, silly mistakes. There's both sides of this.

It would suggest things that just don't compile in ways that are really obvious to anyone who takes a moment to read it. And it would also make really impressive cognitive leaps where it would suggest things that, yes, that is the direction I was heading, and it would have taken me several minutes to explain it to someone, and it got there immediately.

It would suggest things that just don't compile in ways that are really obvious to anyone who takes a moment to read it. And it would also make really impressive cognitive leaps where it would suggest things that, yes, that is the direction I was heading, and it would have taken me several minutes to explain it to someone, and it got there immediately.

And so I spent quite a lot of time working on code completion systems with the goal of improving them by focusing on a particular programming language. And we've made some good progress there. We actually hope to demonstrate some of that publicly soon, like in the next few weeks, probably in this sketch.dev thing that we've been building.

And so I spent quite a lot of time working on code completion systems with the goal of improving them by focusing on a particular programming language. And we've made some good progress there. We actually hope to demonstrate some of that publicly soon, like in the next few weeks, probably in this sketch.dev thing that we've been building.

We'll integrate it so that people can see it and give it a try. But so those models are interesting because they're not the LLM experience that most users have. Like when everyone talks about AI today, they talk about ChatGPT or Claude or these chat-based systems. The thing I really, really like about the original Copilot code completion model is it's not a chat system.

We'll integrate it so that people can see it and give it a try. But so those models are interesting because they're not the LLM experience that most users have. Like when everyone talks about AI today, they talk about ChatGPT or Claude or these chat-based systems. The thing I really, really like about the original Copilot code completion model is it's not a chat system.

It's a different user interface experience for the knowledge in an LLM. And that's really a lot of fun. And in fact, the technology is a little bit different too. There's a concept in the model space called fill in the middle where a model is taught a few extra tokens that don't exist in a standard chat model. With fill in the middle, which is a lot of fun, a model is taught a few extra tokens.

It's a different user interface experience for the knowledge in an LLM. And that's really a lot of fun. And in fact, the technology is a little bit different too. There's a concept in the model space called fill in the middle where a model is taught a few extra tokens that don't exist in a standard chat model. With fill in the middle, which is a lot of fun, a model is taught a few extra tokens.

And it's taught a prefix token, a suffix token, and a middle token. And what you do is you feed in as a prompt to the model the file you're in. And everything, all the characters before where the cursor is, get fed right after a prefix token. So you feed in prefix, all the characters of the file. Then you feed in suffix, and you feed in all the tokens after the cursor.

And it's taught a prefix token, a suffix token, and a middle token. And what you do is you feed in as a prompt to the model the file you're in. And everything, all the characters before where the cursor is, get fed right after a prefix token. So you feed in prefix, all the characters of the file. Then you feed in suffix, and you feed in all the tokens after the cursor.

And then you feed in the middle token, and then you feed in whatever goes into the middle to complete it. And that's the prompt structure for one of these models. And then the model keeps completing the thing that you fed in. It writes the next characters. And you train the model by taking existing code.

And then you feed in the middle token, and then you feed in whatever goes into the middle to complete it. And that's the prompt structure for one of these models. And then the model keeps completing the thing that you fed in. It writes the next characters. And you train the model by taking existing code.

There's a few papers on how these models are trained because Meta published one of these models. Google published one of these models under the Gemma brand. There's a few others out there. There's one from Quinn and some other derived ones.

There's a few papers on how these models are trained because Meta published one of these models. Google published one of these models under the Gemma brand. There's a few others out there. There's one from Quinn and some other derived ones.

And you take existing code files, you choose some section of code, you mark everything before it as the prefix, everything after it as the suffix, and you fill in everything after it as the middle. And that's your training data. You generate a lot of that by taking existing code and breaking it up into these files randomly.

And you take existing code files, you choose some section of code, you mark everything before it as the prefix, everything after it as the suffix, and you fill in everything after it as the middle. And that's your training data. You generate a lot of that by taking existing code and breaking it up into these files randomly.