Glauber Costa
๐ค SpeakerAppearances Over Time
Podcast Appearances
If you want to import a crate that does whatever crazy stuff to implement callers in the CLI, that's fine. But for the core of the database, we try not to import anything that could potentially do I.O. because we want to make sure that everything goes through the simulator.
If you want to import a crate that does whatever crazy stuff to implement callers in the CLI, that's fine. But for the core of the database, we try not to import anything that could potentially do I.O. because we want to make sure that everything goes through the simulator.
It does.
It does.
Yeah, so again, the simulator itself, you just write a couple of scenarios. And again, it's not truly magic because you do have to write the simulator, right? So the simulator just writes, imagine, for example, a workload, and the workload is like generate a couple of queries and then write, etc. But the simulator will include a fuzzing element to that.
Yeah, so again, the simulator itself, you just write a couple of scenarios. And again, it's not truly magic because you do have to write the simulator, right? So the simulator just writes, imagine, for example, a workload, and the workload is like generate a couple of queries and then write, etc. But the simulator will include a fuzzing element to that.
So we will, instead of like generate this query that I wrote in the unit test, you give it almost like a query generator and then it starts generating random queries. Then you start in the simulator itself. What happens, imagine this, if you want to write to a file in any software, you would call a operating system API, write.
So we will, instead of like generate this query that I wrote in the unit test, you give it almost like a query generator and then it starts generating random queries. Then you start in the simulator itself. What happens, imagine this, if you want to write to a file in any software, you would call a operating system API, write.
If we're talking about Rust, it's like a FS write, and then you write to the operating system. When you're writing software for DST, you don't do this. You have your own IO interface abstraction, and then all of your IO goes through this. So when you're running this in production mode, your abstraction for write just called the operating system write.
If we're talking about Rust, it's like a FS write, and then you write to the operating system. When you're writing software for DST, you don't do this. You have your own IO interface abstraction, and then all of your IO goes through this. So when you're running this in production mode, your abstraction for write just called the operating system write.
But when you're running in deterministic simulation testing mode, your abstraction for write runs the simulator code that will start injecting failures into this. And again, injecting failures in a deterministic way, because then if the query fails, because you injected a IO failure at that moment, you will be able to replay that session piece by piece.
But when you're running in deterministic simulation testing mode, your abstraction for write runs the simulator code that will start injecting failures into this. And again, injecting failures in a deterministic way, because then if the query fails, because you injected a IO failure at that moment, you will be able to replay that session piece by piece.
Is that the guy that, is that the guy that. It's barely inconvenient. Very easy, barely inconvenient. That's right. Super easy. I love that guy, man. Wow, wow, wow. Super easy, barely inconvenient. Wow, wow, wow.
Is that the guy that, is that the guy that. It's barely inconvenient. Very easy, barely inconvenient. That's right. Super easy. I love that guy, man. Wow, wow, wow. Super easy, barely inconvenient. Wow, wow, wow.
So how do you rewrite SQLite with the confidence that it will actually have the level of trust that SQLite acquired? It's super easy, barely an inconvenience, right? All you have to do is write the determinants.
So how do you rewrite SQLite with the confidence that it will actually have the level of trust that SQLite acquired? It's super easy, barely an inconvenience, right? All you have to do is write the determinants.
First of all, compatibility in a project like this, and this is the experience that we gained at Scylla when we were rewriting Apache Cassandra. Compatibility has to be like a one-way street, right? You don't want to shackle yourself and say, I will always... Because sometimes, for example, to implement a new feature, you have to do it in a different way.
First of all, compatibility in a project like this, and this is the experience that we gained at Scylla when we were rewriting Apache Cassandra. Compatibility has to be like a one-way street, right? You don't want to shackle yourself and say, I will always... Because sometimes, for example, to implement a new feature, you have to do it in a different way.
So it's usually something, hey, we're going to offer you the same feature set as SQLite. We're going to read your SQLite files. We're going to execute your SQLite code. And if you're not using any of the specific features that we have, we can generate SQLite files as well. But the moment you start using new features that are only present in your implementation, it's impossible to be compatible.
So it's usually something, hey, we're going to offer you the same feature set as SQLite. We're going to read your SQLite files. We're going to execute your SQLite code. And if you're not using any of the specific features that we have, we can generate SQLite files as well. But the moment you start using new features that are only present in your implementation, it's impossible to be compatible.