Jeff Kao

Maybe you want to increment something.

So you have to understand that the state of your database looks like this.

You're going to apply this operation.

What happens if your database script fails?

Then you've got to code in this concept of idempotency, where you can rerun the script many times.

And that's a lot of things to reason about.

So...

In terms of a deployment story, for sure, we wanted something that was very simple to deploy.

And Rust just gives you a simple binary.

And we essentially compile it in CI and essentially ship up the binary to our servers.

But also the data assets, we wanted it all to be self-contained within the same server.

And even the way that we process it, actually, we rebuild the data sets from scratch.

We never backfill anything.

We just take all the raw sources and then, you know, distribute the compute with something like Spark and essentially compile it to, you know, some data assets.

And there are some things that are more specific to, you know, the data formats that we use on our server.

So we process those later and then we ship those data assets to all of these boxes.

And so, you know, what that means is because everything is self-contained, it's actually very trivial to roll back, which is something that you might not be able to do if you had, say, like a third, you know, like if very simple, like two-tier architecture, even a web app and a, say, SQL server.

If you do a data migration,

You have to reason about the data migration rollback and forward, as well as the binary data rollback and forward.

But with our new service, it all sort of goes in one package and lockstep.