David Shu

by randomly inserting a cursor, then you've taught a model how to use these extra characters and how to complete them. And so it's not a chat model at all. It's sort of a sequence-to-sequence model. It's a ton of fun. And the advantage of these systems is they're very fast compared to chat models.

by randomly inserting a cursor, then you've taught a model how to use these extra characters and how to complete them. And so it's not a chat model at all. It's sort of a sequence-to-sequence model. It's a ton of fun. And the advantage of these systems is they're very fast compared to chat models.

And that's the key to the whole code completion product experience is you want your code completion within a couple hundred milliseconds of you typing a character. Whereas if you actually time Claude or you time one of the open AI models, they're very slow. Like they take a good minute to give you a result. And there's a lot of UI tricks in hiding that minute.

And that's the key to the whole code completion product experience is you want your code completion within a couple hundred milliseconds of you typing a character. Whereas if you actually time Claude or you time one of the open AI models, they're very slow. Like they take a good minute to give you a result. And there's a lot of UI tricks in hiding that minute.

They move the text around on the screen, then they stream it in. Yeah, it's very clever.

They move the text around on the screen, then they stream it in. Yeah, it's very clever.

Yeah, exactly. You can really feel it with the new reasoning models, O1, these things, because there's this pause at the beginning. It hurts.

Yeah, exactly. You can really feel it with the new reasoning models, O1, these things, because there's this pause at the beginning. It hurts.

Yeah, it is a ton of fun to watch. I agree. And it is a lot of insight into how the models work, too. Because the insides of the models are a large number of floating point numbers holding intermediate state. And it's very hard to get insight into those. But the words, you can read them. You can make some sense of them. So code completion is, I think, extremely useful to programmers.

Yeah, it is a ton of fun to watch. I agree. And it is a lot of insight into how the models work, too. Because the insides of the models are a large number of floating point numbers holding intermediate state. And it's very hard to get insight into those. But the words, you can read them. You can make some sense of them. So code completion is, I think, extremely useful to programmers.

It varies a lot depending on what you're writing and how experienced models are with it. And just... how sort of out on the edge of programming you are. If you're really out in the weeds, the models can get less useful. I used a model for writing a big chunk of AVX assembly a few months ago. And the model was both very good at it and very bad at it simultaneously.

It varies a lot depending on what you're writing and how experienced models are with it. And just... how sort of out on the edge of programming you are. If you're really out in the weeds, the models can get less useful. I used a model for writing a big chunk of AVX assembly a few months ago. And the model was both very good at it and very bad at it simultaneously.

And it was very different from the typical asking a model to help with programming experience. it would constantly get the order operations wrong or overcomplicate things or misunderstand. It was a very different experience than typical programming.

And it was very different from the typical asking a model to help with programming experience. it would constantly get the order operations wrong or overcomplicate things or misunderstand. It was a very different experience than typical programming.

I used all of them for that. Okay. And this is what I meant by I'm spending a lot of time actively exploring the space. I'm putting far too much work into exploring the model space as I do work.

I used all of them for that. Okay. And this is what I meant by I'm spending a lot of time actively exploring the space. I'm putting far too much work into exploring the model space as I do work.

I can't advise people to use them all. You know, that's a bunch of them. Yeah. It's a, and this I think is the big problem. And you know, You mentioned that most programmers are probably using this. As far as we can tell, not one fifth of programmers are using these tools today.

I can't advise people to use them all. You know, that's a bunch of them. Yeah. It's a, and this I think is the big problem. And you know, You mentioned that most programmers are probably using this. As far as we can tell, not one fifth of programmers are using these tools today.

Through surveys. A couple of people have done surveys of programmers and it seems to come back that most people are not using these tools yet. Wow. Which is both shocking to me because they're so useful and also makes a lot of sense because it's a lot of work figuring out how to use them.

Through surveys. A couple of people have done surveys of programmers and it seems to come back that most people are not using these tools yet. Wow. Which is both shocking to me because they're so useful and also makes a lot of sense because it's a lot of work figuring out how to use them.