Arvid Lunnemark

And even for today's models, even when you have long context, filling out the entire context window means that it's slower. It means that sometimes the model actually gets confused and some models get more confused than others. And we have this one system internally that we call pre-empt, which helps us with that a little bit.

And I think it was built for the era before where we had 8,000 token context windows. And it's a little bit similar to when you're making a website. You want it to work on mobile. You want it to work on a desktop screen. And you have this dynamic information, which you don't have, for example, if you're designing a print magazine. You know exactly where you can put stuff.

But when you have a website or when you have a prompt, you have these inputs. And then you need to format them to always work. Even if the input is really big, then you might have to cut something down. And so the idea was, okay, let's take some inspiration. What's the best way to design websites?

Well, the thing that we really like is React and the declarative approach where you use JSX in JavaScript, and then you declare, this is what I want, and I think this has higher priority, or this has higher z-index than something else. And then you have this rendering engine. In web design, it's like Chrome, and in our case, it's a preempt renderer, which then fits everything onto the page.

And as you clearly decide what you want, and then it figures out what you want. And so we have found that to be quite helpful. And I think the role of it has sort of shifted over time, where initially it was to fit to these small context windows. Now it's really useful because it helps us with...

splitting up the data that goes into the prompt and the actual rendering of it and so it's easier to debug because you can change the rendering of the prompt and then try it on old prompts because you have the raw data that went into their prompt and then you can see did my change actually improve it for for like this entire eval set so do you literally prompt with jsx Yes, yes.

So it kind of looks like React. There are components. We have one component that's a file component, and it takes in the cursor. Usually there's one line where the cursor is in your file, and that's probably the most important line because that's the one you're looking at. And so then you can give priorities.

So that line has the highest priority, and then you subtract one for every line that is farther away. And then eventually when it's rendered, it figures out how many lines can actually fit, and it centers around that thing.

I think our goal is kind of that you should just do whatever is the most natural thing for you. And then we, our job is to figure out how do we actually like retrieve the relative event thing so that your thing actually makes sense.

We think agents are really, really cool. I think agents is like... It's like it resembles sort of like a human. It's sort of like you can kind of feel that you're getting closer to AGI because you see a demo where it acts as a human would. And it's really, really cool. I think... agents are not yet super useful for many things. I think we're getting close to where they will actually be useful.

And so I think there are certain types of tasks where having an agent would be really nice. I would love to have an agent. For example, we have a bug where you sometimes can't command C and command V inside our chat input box, and that's a task that's super well specified. I just want to say in two sentences, this does not work, please fix it.

And then I would love to have an agent that just goes off, does it, and then a day later I come back and I review the thing.

Yeah, it finds the right files, it tries to reproduce the bug, it fixes the bug, and then it verifies that it's correct. And this could be a process that takes a long time. And so I think I would love to have that. And then I think a lot of programming, there is often this belief that agents will take over all of programming.

I don't think we think that that's the case because a lot of programming, a lot of the value is in iterating or you don't actually want to specify something upfront because you don't really know what you want until you've seen an initial version and then you want to iterate on that and then you provide more information.

And so for a lot of programming, I think you actually want a system that's instant that gives you an initial version instantly back and then you can iterate super, super quickly.

Yeah.

I think so. I think that would be really cool. For certain types of programming, it would be really cool.

Yeah, we aren't actively working on it right now. But it's definitely like, we want to make the programmer's life easier and more fun. And some things are just really tedious and you need to go through a bunch of steps and you want to delegate that to an agent. And then some things you can actually have an agent in the background while you're working.

Like, let's say you have a PR that's both backend and frontend, and you're working in the frontend, and then you can have a background agent that does some work and figure out kind of what you're doing. And then when you get to the backend part of your PR, then you have some like initial piece of code that you can iterate on. And so that would also be really cool.

It's a pain. It's a pain that we're feeling and we're working on fixing it.