Cian Butler

One of the bottlenecks I found was we were wasting cycles doing work for connections that had already closed.

Oh, wow.

Yeah.

Why is that?

It's a little to do with our queuing model and a little bit to do with request management in uWhiskey, the process we were using.

Effectively, if a request had sat in the queue for too long, it would be handed over to uWhiskey.

uWhiskey would do it and it would time out in the upstream because it had been processing for longer than a minute.

But there's no way to cancel the request once it's in flow.

We would benefit from it because we do all the work and cache the result.

So the request would have been retried and would be in the queue.

And by the time it gets to the front of the queue, all its results are cached.

So it was a nasty flow, but we kind of optimized for it.

But I thought to myself, this feels insane.

There's no guarantee that that request is going to be cached or it's going to be re-driven.

Most of the time it is.

We're dealing with some of the... I think someone recently described some of our clients as some of the best and worst clients in the world because they're designed for public infrastructure since they're all package management clients.

So they have a lot of retries, but they have a lot of weird formats.

So

We're dealing with some of the best and worst clients.

So we know a lot of things are going to be like retried and attempted again.