Menu
Sign In Search Podcasts Libraries Charts People & Topics Add Podcast API Blog Pricing

Martin Kleppmann

๐Ÿ‘ค Speaker
607 total appearances

Appearances Over Time

Podcast Appearances

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

Where I think the cloud has helped quite a bit is actually at the lower end of scaling down.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

If you want to have a very lightweight service that processes only a small number of requests,

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

What we've got with serverless systems being able to very quickly spin up and spin down an instance, very lightweight.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

That's quite a good innovation that has enabled those very low scale services.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

And that's something that would be much harder to do without cloud services because you would have to statically allocate a certain amount of memory and certain CPU resources to a particular virtual machine.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

I think it's probably less of an effect of cloud and more of just hardware getting more powerful that, oh, actually, like a big machine nowadays can do a lot on a big machine.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

And that means that more and more workloads you can just run on a single machine.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

And that is sufficient actually to achieve quite significant scale already.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

There's still concerns of how to actually efficiently make use of hundreds of CPU cores that you have on a single machine.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

So there's still parallelism is still a required thing to think about there.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

And sharding is one way of achieving parallelism.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

But at least this sort of sharding across multiple machines has maybe become less of a pressing issue just because more and more workloads can just run on a single machine.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

Some people still have very large scale workloads that do have to be sharded across multiple machines.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

So it's not going away entirely.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

And replication is still relevant even at smaller scales because that's for fault tolerance.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

That's not for scalability.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

Yeah, the whole idea of this chapter is that in distributed systems theory, there are certain things that we tend to assume.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

Like, for example, we just assume that there's no upper bound on how long it might take for a message to go over the network.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

So you send a message, it might arrive within 100 microseconds, or it might take 10 years.

The Pragmatic Engineer
Designing Data-intensive Applications with Martin Kleppmann

And