Martin Kleppmann
๐ค SpeakerAppearances Over Time
Podcast Appearances
Where I think the cloud has helped quite a bit is actually at the lower end of scaling down.
If you want to have a very lightweight service that processes only a small number of requests,
What we've got with serverless systems being able to very quickly spin up and spin down an instance, very lightweight.
That's quite a good innovation that has enabled those very low scale services.
And that's something that would be much harder to do without cloud services because you would have to statically allocate a certain amount of memory and certain CPU resources to a particular virtual machine.
I think it's probably less of an effect of cloud and more of just hardware getting more powerful that, oh, actually, like a big machine nowadays can do a lot on a big machine.
And that means that more and more workloads you can just run on a single machine.
And that is sufficient actually to achieve quite significant scale already.
There's still concerns of how to actually efficiently make use of hundreds of CPU cores that you have on a single machine.
So there's still parallelism is still a required thing to think about there.
And sharding is one way of achieving parallelism.
But at least this sort of sharding across multiple machines has maybe become less of a pressing issue just because more and more workloads can just run on a single machine.
Some people still have very large scale workloads that do have to be sharded across multiple machines.
So it's not going away entirely.
And replication is still relevant even at smaller scales because that's for fault tolerance.
That's not for scalability.
Yeah, the whole idea of this chapter is that in distributed systems theory, there are certain things that we tend to assume.
Like, for example, we just assume that there's no upper bound on how long it might take for a message to go over the network.
So you send a message, it might arrive within 100 microseconds, or it might take 10 years.
And