Marc Brooker

Similarly, if you look at a more, let's say, classical relational database design like Aurora, the Aurora leader is constantly telling the potential failover targets, here's something you should cache, here's something you should cache, here's something you should cache.

So when a failover happens, the cache is warm on the failover target.

And so those are the kinds of things that you can do to avoid those modalities.

But in general,

I wouldn't extract this as a rule or say that this applies 100% of the time.

But in general, I prefer to see the teams around me avoiding caching where possible.

I prefer patterns where you have a, let's say, complete materialized view of the data if you need very fast access to it.

Especially if it's slow moving, just pull it down onto your local machine and work with it in memory.

If it's only being updated once a week, who cares?

Just make lots of copies of it.

So that's one pattern.

Or use a scalable backend, DSQL or DynamoDB or whatever your favorite scalable database is, and keep your database vendor honest about getting to the scale and performance you need rather than putting a cache in front of things.

So caching isn't a bad pattern, but it is a pattern with some...

significant downsides that are really best avoided.

Yeah, it's not super common, right?

Like you might go years without seeing something like that.

But if you look across

the biggest, most impactful system postmortems across the industry, I would say that these kinds of metastable failures have been an underlying cause in probably a majority of them.

And it's super important that as an industry and as a community of practice, we understand those things deeply because also those cases where these do happen,

you know, tend to be larger scale issues, longer recovery time issues, and more complex to fix issues, right?