Aman Sanger

Yeah. So like the basic, the size of your KV cache is both the size of all your prompts multiplied by the number of prompts being processed in parallel. So you could increase either those dimensions, right? The batch size or the size of your prompts without degrading the latency of generating tokens.

One maybe hacky but interesting idea that I like is holding a lock on saving. And so basically, you can then have the language model kind of hold the lock on saving to disk.

One maybe hacky but interesting idea that I like is holding a lock on saving. And so basically, you can then have the language model kind of hold the lock on saving to disk.

One maybe hacky but interesting idea that I like is holding a lock on saving. And so basically, you can then have the language model kind of hold the lock on saving to disk.

And then instead of you operating in the ground truth version of the files that are saved to disk, you actually are operating what was the shadow workspace before and these unsaved things that only exist in memory that you still get linter errors for and you can

And then instead of you operating in the ground truth version of the files that are saved to disk, you actually are operating what was the shadow workspace before and these unsaved things that only exist in memory that you still get linter errors for and you can

And then instead of you operating in the ground truth version of the files that are saved to disk, you actually are operating what was the shadow workspace before and these unsaved things that only exist in memory that you still get linter errors for and you can

code in and then when you try to maybe run code it's just like there's a small warning that there's a lock and then you kind of will take back the lock from the language server if you're trying to do things concurrently or from the shadow workspace if you're trying to do things concurrently that's such an exciting future by the way it's a bit of a tangent but like to allow a model to change files

code in and then when you try to maybe run code it's just like there's a small warning that there's a lock and then you kind of will take back the lock from the language server if you're trying to do things concurrently or from the shadow workspace if you're trying to do things concurrently that's such an exciting future by the way it's a bit of a tangent but like to allow a model to change files

code in and then when you try to maybe run code it's just like there's a small warning that there's a lock and then you kind of will take back the lock from the language server if you're trying to do things concurrently or from the shadow workspace if you're trying to do things concurrently that's such an exciting future by the way it's a bit of a tangent but like to allow a model to change files

Yeah. And I think there may be different versions of like run ability where you For the simple things where you're doing things in the span of a few minutes on behalf of the user as they're programming, it makes sense to make something work locally in their machine.

Yeah. And I think there may be different versions of like run ability where you For the simple things where you're doing things in the span of a few minutes on behalf of the user as they're programming, it makes sense to make something work locally in their machine.

Yeah. And I think there may be different versions of like run ability where you For the simple things where you're doing things in the span of a few minutes on behalf of the user as they're programming, it makes sense to make something work locally in their machine.

I think for the more aggressive things where you're making larger changes that take longer periods of time, you'll probably want to do this in some sandbox remote environment. And that's another incredibly tricky problem of how do you

I think for the more aggressive things where you're making larger changes that take longer periods of time, you'll probably want to do this in some sandbox remote environment. And that's another incredibly tricky problem of how do you

I think for the more aggressive things where you're making larger changes that take longer periods of time, you'll probably want to do this in some sandbox remote environment. And that's another incredibly tricky problem of how do you

exactly reproduce or mostly reproduce to the point of it being effectively equivalent for running code the user's environment with this remote remote sandbox i'm curious what kind of agency you want for for coding did you want them to find bugs do you want them to like implement new features like what agency you want

exactly reproduce or mostly reproduce to the point of it being effectively equivalent for running code the user's environment with this remote remote sandbox i'm curious what kind of agency you want for for coding did you want them to find bugs do you want them to like implement new features like what agency you want

exactly reproduce or mostly reproduce to the point of it being effectively equivalent for running code the user's environment with this remote remote sandbox i'm curious what kind of agency you want for for coding did you want them to find bugs do you want them to like implement new features like what agency you want

Yeah. I mean, it's really interesting that these models are so bad at bug finding when just naively prompted to find a bug. They're incredibly poorly calibrated.