Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Pricing

John Gallagher

👤 Person
314 total appearances

Appearances Over Time

Podcast Appearances

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

You should definitely not look at doing everything all at once. As I think we can all agree in software, doing everything all at once is a recipe for disaster, no matter what you're doing.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

You should definitely not look at doing everything all at once. As I think we can all agree in software, doing everything all at once is a recipe for disaster, no matter what you're doing.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

There are vendors that tell you that you can do that. Whether you actually can or not is a different matter. Spoiler alert, you can't. So I just want to back up a little bit and talk about the feelings because I think it's the feelings that is where all of this start for me.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

There are vendors that tell you that you can do that. Whether you actually can or not is a different matter. Spoiler alert, you can't. So I just want to back up a little bit and talk about the feelings because I think it's the feelings that is where all of this start for me.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

So I got into observability and it's funny because for the first kind of year of my journey doing this, I didn't even realize I was doing observability. I'd heard about this observability thing and it was out there in the universe. Okay. Maybe I should learn that. I should learn that. And I kept using the should. I should learn this. I should have loads of other stuff to do.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

So I got into observability and it's funny because for the first kind of year of my journey doing this, I didn't even realize I was doing observability. I'd heard about this observability thing and it was out there in the universe. Okay. Maybe I should learn that. I should learn that. And I kept using the should. I should learn this. I should have loads of other stuff to do.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

I've got loads of other things. I don't know what it is. I know it comes from controls here, and there's a Wikipedia page that's really complex and really confusing. Whatever. I've got real work to do. But what I know is that I kept coming across these bugs in Bugsnag, Sentry, Airbrake. Choose your error reporting tool. They all help you to a degree, but they're not a silver bullet.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

I've got loads of other things. I don't know what it is. I know it comes from controls here, and there's a Wikipedia page that's really complex and really confusing. Whatever. I've got real work to do. But what I know is that I kept coming across these bugs in Bugsnag, Sentry, Airbrake. Choose your error reporting tool. They all help you to a degree, but they're not a silver bullet.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

And I kept coming across these defects over and over, and the story was exactly the same. Come across a defect, I'd see the stack trace in the error reporting tool, and I would look at it, and first emotion right out the gate, complete confusion. What is going on here? No idea. So I dig a little bit into the code. I dig a little bit into the stack trace. So it's coming from here.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

And I kept coming across these defects over and over, and the story was exactly the same. Come across a defect, I'd see the stack trace in the error reporting tool, and I would look at it, and first emotion right out the gate, complete confusion. What is going on here? No idea. So I dig a little bit into the code. I dig a little bit into the stack trace. So it's coming from here.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

And this thing is nil. Classic, right? This thing is nil. Where was it being passed in as nil? I don't know. So now I'm like, well, I can't just say I can't fix this. So I now have to, well, do what exactly? I don't have any information to go off. Well, I guess we'll do that bug later. Let's look at the next one. And this just kept happening.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

And this thing is nil. Classic, right? This thing is nil. Where was it being passed in as nil? I don't know. So now I'm like, well, I can't just say I can't fix this. So I now have to, well, do what exactly? I don't have any information to go off. Well, I guess we'll do that bug later. Let's look at the next one. And this just kept happening.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

And I would find myself going through all the bugs in the backlog and I couldn't fix any of them. And I just wasted four hours looking at things, asking questions that I couldn't explain, looking at things I didn't understand. And for years, I thought the problem was with me. I honestly thought I'm just not smart enough. I'm not a good engineer, blah, blah, blah, blah, blah.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

And I would find myself going through all the bugs in the backlog and I couldn't fix any of them. And I just wasted four hours looking at things, asking questions that I couldn't explain, looking at things I didn't understand. And for years, I thought the problem was with me. I honestly thought I'm just not smart enough. I'm not a good engineer, blah, blah, blah, blah, blah.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

Bug fixing just isn't really my thing. I'm just not really good at it. And then after many, many years of this, I was in a company, and I just got really sick of this. We just released a brand-new app, and it was a customer account app. And we were getting all these weird bug reports, people saying I can't log in, people saying I can't reset my password.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

Bug fixing just isn't really my thing. I'm just not really good at it. And then after many, many years of this, I was in a company, and I just got really sick of this. We just released a brand-new app, and it was a customer account app. And we were getting all these weird bug reports, people saying I can't log in, people saying I can't reset my password.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

And every time we did this, we would add a little bit of kind of this ad hoc logging and then put the bug back in the backlog. And then it would come up again and come up again. And after a while, I was just like, this is just, this is ridiculous. We're highly paid engineers. This is not a better way.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

And every time we did this, we would add a little bit of kind of this ad hoc logging and then put the bug back in the backlog. And then it would come up again and come up again. And after a while, I was just like, this is just, this is ridiculous. We're highly paid engineers. This is not a better way.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

So then I started looking into, we were using Kibana at the time, or rather I should say we were not using Kibana at the time. Kibana was there, we were paying for it. And I was like, I've heard this is something to do with logging. So where do we do our logging? People like Kibana. I have no idea what this even is. Let's open it up. And there was just all of this trash. all of this rubbish.

Ruby Rogues
Practical Observability: Logging, Tracing, and Metrics for Better Debugging - RUBY 656

So then I started looking into, we were using Kibana at the time, or rather I should say we were not using Kibana at the time. Kibana was there, we were paying for it. And I was like, I've heard this is something to do with logging. So where do we do our logging? People like Kibana. I have no idea what this even is. Let's open it up. And there was just all of this trash. all of this rubbish.