John Gallagher

Sure. Thanks for having me on. My name is John Gallagher, and I am a senior engineer at I had a company called BiggerPockets, and we teach how to invest in real estate based in the US. And I also run my own business on the side called Joyful Programming to introduce more joy to the world of programming. And I'm on today to talk a bit about observability, which is one of my many passions.

Sure. Thanks for having me on. My name is John Gallagher, and I am a senior engineer at I had a company called BiggerPockets, and we teach how to invest in real estate based in the US. And I also run my own business on the side called Joyful Programming to introduce more joy to the world of programming. And I'm on today to talk a bit about observability, which is one of my many passions.

I'm a bit of a polymath. This is one of the things that is really, really important to me and I'm passionate about. I'm particularly passionate about introducing this into Rails apps. So thanks for having me on.

I'm a bit of a polymath. This is one of the things that is really, really important to me and I'm passionate about. I'm particularly passionate about introducing this into Rails apps. So thanks for having me on.

Well, um, I don't actually think anybody should care about observability and I don't care about observability as a thing because it's just a means to an end. And what's the actual goal. Um, Doesn't matter how you get there, but the goal is being able to, number one, understand your Rails app in production, and number two, be able to ask unusual questions.

Well, um, I don't actually think anybody should care about observability and I don't care about observability as a thing because it's just a means to an end. And what's the actual goal. Um, Doesn't matter how you get there, but the goal is being able to, number one, understand your Rails app in production, and number two, be able to ask unusual questions.

Not questions that you've thought of a day, two days, three weeks ago, because that's not really very useful or interesting. If we knew exactly the questions to ask in the future of our apps, everything would be easy. Just be like, how many 200s have we had in the last week? It's kind of a boring question to ask. Maybe a bit useful. I find the more obvious the question, the less useful it is.

Not questions that you've thought of a day, two days, three weeks ago, because that's not really very useful or interesting. If we knew exactly the questions to ask in the future of our apps, everything would be easy. Just be like, how many 200s have we had in the last week? It's kind of a boring question to ask. Maybe a bit useful. I find the more obvious the question, the less useful it is.

So observability is the practice of making a black box system more transparent. So I like to think of it, imagine your entire Rails app, all the hosting, everything to do with that app is wrapped up in an opaque black box. And somebody says, how does it work? And why is this thing going wrong? You would have no hope of understanding it. If the box is completely translucent,

So observability is the practice of making a black box system more transparent. So I like to think of it, imagine your entire Rails app, all the hosting, everything to do with that app is wrapped up in an opaque black box. And somebody says, how does it work? And why is this thing going wrong? You would have no hope of understanding it. If the box is completely translucent,

And you can see everything, which of course is completely impossible in software. But in theory, you'd have this completely translucent box and you can ask all these questions and you get instant answers. That's like 100% observability. And of course, that is absolutely impossible.

And you can see everything, which of course is completely impossible in software. But in theory, you'd have this completely translucent box and you can ask all these questions and you get instant answers. That's like 100% observability. And of course, that is absolutely impossible.

And so what we're trying to do with observability is understand what is going on, not just when it goes wrong, although that's the obvious use case, is we have an incident, the most critical point where observability comes into play is an exact scenario that I landed in two weeks into a new role I had. So it was two weeks in, the site had gone down,

And so what we're trying to do with observability is understand what is going on, not just when it goes wrong, although that's the obvious use case, is we have an incident, the most critical point where observability comes into play is an exact scenario that I landed in two weeks into a new role I had. So it was two weeks in, the site had gone down,

I am in the UK, and the rest of my team were in the US, and there were two other engineers in my time zone. And all of us had been at the company for a total of five weeks. So we've got this app. It's down. It's on fire. And we need to put the fire out. And the three of us looked at each other. We were like, should we just restart the dynos? Yeah. So we restarted the dynos.

I am in the UK, and the rest of my team were in the US, and there were two other engineers in my time zone. And all of us had been at the company for a total of five weeks. So we've got this app. It's down. It's on fire. And we need to put the fire out. And the three of us looked at each other. We were like, should we just restart the dynos? Yeah. So we restarted the dynos.

We crossed our fingers. And it was pure luck that the app came back up. That is the exact opposite of what we want. And we've now moved to a situation where we can ask our app a whole load of very unusual questions. And we will get an answer to that. Why are there a peak of 404s on iOS at 3 a.m.? Looks like a lot of them are coming from this IP address.

We crossed our fingers. And it was pure luck that the app came back up. That is the exact opposite of what we want. And we've now moved to a situation where we can ask our app a whole load of very unusual questions. And we will get an answer to that. Why are there a peak of 404s on iOS at 3 a.m.? Looks like a lot of them are coming from this IP address.

Okay, what's that IP address doing on the site? Okay, interesting. How many users are using that IP address? Five. So only five people are using it. So that's the point of observability to me, to be able to ask unusual questions that you haven't thought of already,

Okay, what's that IP address doing on the site? Okay, interesting. How many users are using that IP address? Five. So only five people are using it. So that's the point of observability to me, to be able to ask unusual questions that you haven't thought of already,