Neil Patel

And so on our side, we tried to plan it in that way, which was this was OK to scale later, maybe because you can just throw machines at it initially and you don't need to worry as much. Or maybe it's just going to be a problem and we have to defer it for a while until someone's actually using it in angle, right? Instead of just us theoretically thinking people use it.

On the flip side, I touched on this with S3 and Lambda, thinking about how are we going to scale our ingest so we can write as many objects as we want to write into S3? What is the partitioning that works best for S3 on read? How do we avoid getting hot partitions, etc.? There's one thing changing a schema in a table, something really different changing the way a data store looks up blocks.

On the flip side, I touched on this with S3 and Lambda, thinking about how are we going to scale our ingest so we can write as many objects as we want to write into S3? What is the partitioning that works best for S3 on read? How do we avoid getting hot partitions, etc.? There's one thing changing a schema in a table, something really different changing the way a data store looks up blocks.

On the flip side, I touched on this with S3 and Lambda, thinking about how are we going to scale our ingest so we can write as many objects as we want to write into S3? What is the partitioning that works best for S3 on read? How do we avoid getting hot partitions, etc.? There's one thing changing a schema in a table, something really different changing the way a data store looks up blocks.

for querying, right? It's one of those things where you don't want to come in afterwards and actually play around too much with that. You can always make changes but you don't want to completely change it, especially once you have users. We anticipated those issues, but we definitely ran into things which we didn't. We found the edges of Lambda.

for querying, right? It's one of those things where you don't want to come in afterwards and actually play around too much with that. You can always make changes but you don't want to completely change it, especially once you have users. We anticipated those issues, but we definitely ran into things which we didn't. We found the edges of Lambda.

for querying, right? It's one of those things where you don't want to come in afterwards and actually play around too much with that. You can always make changes but you don't want to completely change it, especially once you have users. We anticipated those issues, but we definitely ran into things which we didn't. We found the edges of Lambda.

We found the edges of using S3 in the way that we use it. We were lucky enough to be able to talk to the S3 teams, the Lambda teams, et cetera, and we were able to basically figure out what those issues were, work around them. Our own load testing and things like that got more sophisticated over time.

We found the edges of using S3 in the way that we use it. We were lucky enough to be able to talk to the S3 teams, the Lambda teams, et cetera, and we were able to basically figure out what those issues were, work around them. Our own load testing and things like that got more sophisticated over time.

We found the edges of using S3 in the way that we use it. We were lucky enough to be able to talk to the S3 teams, the Lambda teams, et cetera, and we were able to basically figure out what those issues were, work around them. Our own load testing and things like that got more sophisticated over time.

We started to learn about making sure that we tested not for the glorious, amazing batches the largest organizations were to send us, but also just hundreds and thousands or more of users of one app that's just sending data, individual events coming from the browser directly to Axiom, for instance. We also had to level up our testing, et cetera, as we kept on building Axiom.

We started to learn about making sure that we tested not for the glorious, amazing batches the largest organizations were to send us, but also just hundreds and thousands or more of users of one app that's just sending data, individual events coming from the browser directly to Axiom, for instance. We also had to level up our testing, et cetera, as we kept on building Axiom.

We started to learn about making sure that we tested not for the glorious, amazing batches the largest organizations were to send us, but also just hundreds and thousands or more of users of one app that's just sending data, individual events coming from the browser directly to Axiom, for instance. We also had to level up our testing, et cetera, as we kept on building Axiom.

I think it's the architecture. I think now as we scale, the architecture is really showing off how pluggable it is and how reimagining what one of these data stores could be like by it being just parts inside of a cloud, AWS, Azure, it doesn't really matter. and how we can rip apart, piece it back together, put things in the middle.

I think it's the architecture. I think now as we scale, the architecture is really showing off how pluggable it is and how reimagining what one of these data stores could be like by it being just parts inside of a cloud, AWS, Azure, it doesn't really matter. and how we can rip apart, piece it back together, put things in the middle.

I think it's the architecture. I think now as we scale, the architecture is really showing off how pluggable it is and how reimagining what one of these data stores could be like by it being just parts inside of a cloud, AWS, Azure, it doesn't really matter. and how we can rip apart, piece it back together, put things in the middle.

All the challenges are so naturally challenges of any cloud kind of systems engineer. It's cheap mode at this point where you get to just pick a service from AWS and say, if I put that in the middle, we're going to automatically get something X, Y, Z happen and that solves this problem or allows customers to do something they weren't before.

All the challenges are so naturally challenges of any cloud kind of systems engineer. It's cheap mode at this point where you get to just pick a service from AWS and say, if I put that in the middle, we're going to automatically get something X, Y, Z happen and that solves this problem or allows customers to do something they weren't before.

All the challenges are so naturally challenges of any cloud kind of systems engineer. It's cheap mode at this point where you get to just pick a service from AWS and say, if I put that in the middle, we're going to automatically get something X, Y, Z happen and that solves this problem or allows customers to do something they weren't before.

We had on the whiteboard when we announced that we were pivoting and we did a sprint with the team and we were like, okay, we're going to build this data store. The first couple of things on the whiteboard were essentially the cloud exists with two underlines.