Ryan Worrell
๐ค SpeakerAppearances Over Time
Podcast Appearances
So the fact that everything is in object storage opens up a ton of new possibilities, actually. Basically, the only downside of WarpStream is the fact that the latency is higher. Now, obviously, we're a new company. The product does not have the 13-year maturity of the open-source Kafka project. But just to speak of the operational...
So the fact that everything is in object storage opens up a ton of new possibilities, actually. Basically, the only downside of WarpStream is the fact that the latency is higher. Now, obviously, we're a new company. The product does not have the 13-year maturity of the open-source Kafka project. But just to speak of the operational...
stuff and the cost stuff, the Workstream is a huge win on both of those.
stuff and the cost stuff, the Workstream is a huge win on both of those.
Yeah, so there are a number of projects and products out there that you can buy to give you an object storage interface in essentially any environment. Like there's the open source project, MinIO, and then basically every storage vendor on the market will sell you something with an S3 compatible interface if you're running in a data center environment.
Yeah, so there are a number of projects and products out there that you can buy to give you an object storage interface in essentially any environment. Like there's the open source project, MinIO, and then basically every storage vendor on the market will sell you something with an S3 compatible interface if you're running in a data center environment.
And because we work with S3, GCS, and Azure Blob Storage, we can essentially, you know, I shouldn't say connect to anything. If you had an NFS server, we can even make it work on that too. We don't have any production doing that, and I wouldn't recommend it. I would recommend using the object storage interfaces, but we're pretty flexible in terms of the deployment topology.
And because we work with S3, GCS, and Azure Blob Storage, we can essentially, you know, I shouldn't say connect to anything. If you had an NFS server, we can even make it work on that too. We don't have any production doing that, and I wouldn't recommend it. I would recommend using the object storage interfaces, but we're pretty flexible in terms of the deployment topology.
So I think it would depend on where you're running the compute. If you were storing the data in R2, but you were running compute in AWS, you would get charged a lot of internet transfer as part of that. If you're running your compute in one of the providers that has free peering with R2, then yeah, you would get a nice savings there and you're, you know,
So I think it would depend on where you're running the compute. If you were storing the data in R2, but you were running compute in AWS, you would get charged a lot of internet transfer as part of that. If you're running your compute in one of the providers that has free peering with R2, then yeah, you would get a nice savings there and you're, you know,
You'd be able to move data reliably across, let's say, multiple regions of whatever providers have peered for free with R2 using Workstream.
You'd be able to move data reliably across, let's say, multiple regions of whatever providers have peered for free with R2 using Workstream.
Yeah, I think the demo was Richie's idea. It basically just starts up a producer and a consumer so that you can just see something happening in the console. Like, yeah, it provides you a link. If you would have run that locally on your laptop, we would have opened the link automatically in your browser for you.
Yeah, I think the demo was Richie's idea. It basically just starts up a producer and a consumer so that you can just see something happening in the console. Like, yeah, it provides you a link. If you would have run that locally on your laptop, we would have opened the link automatically in your browser for you.
Yeah, so we even designed the little niceties like that. But the idea behind the demo is basically just to show people that it does something. Kafka is not an exciting technology to demo, so we're kind of limited there. It's even more boring than doing a demo for a relational database or something. But there is another mode that you can run that's called Playground.
Yeah, so we even designed the little niceties like that. But the idea behind the demo is basically just to show people that it does something. Kafka is not an exciting technology to demo, so we're kind of limited there. It's even more boring than doing a demo for a relational database or something. But there is another mode that you can run that's called Playground.
And Playground will let you start a cluster that doesn't have a fake producer and consumer running on it as a demo. It just starts a cluster for you temporarily and makes an account that expires in 24 hours. And you can take that Playground link and you can start...
And Playground will let you start a cluster that doesn't have a fake producer and consumer running on it as a demo. It just starts a cluster for you temporarily and makes an account that expires in 24 hours. And you can take that Playground link and you can start...
multiple nodes, like say one on my laptop and one on yours, and point it at R2, and we can have a cluster that spans our two laptops together. Like my co-founder and I did that before and posted a video of it on Twitter or something like that. But because the data is all in object storage and the compute part is stateless, it's actually, it's not that complicated to do.
multiple nodes, like say one on my laptop and one on yours, and point it at R2, and we can have a cluster that spans our two laptops together. Like my co-founder and I did that before and posted a video of it on Twitter or something like that. But because the data is all in object storage and the compute part is stateless, it's actually, it's not that complicated to do.