Aaron Crow
👤 PersonAppearances Over Time
Podcast Appearances
Like, yes, this incident, it was caused by CrowdStrike, but it's not the bigger picture of the root cause analysis. If I did a root cause analysis on this, the issue, yes, that was the trigger, but that is not the root cause of the problem that caused this bigger issue.
Like, yes, this incident, it was caused by CrowdStrike, but it's not the bigger picture of the root cause analysis. If I did a root cause analysis on this, the issue, yes, that was the trigger, but that is not the root cause of the problem that caused this bigger issue.
Well, I'll give I'll give a last story here. And it's very similar to that. You know, working in these places, operators have we do what's called operator rounds where they walk around with some kind of handheld. It could be a piece of paper or whatever. In this particular area, it was a it was a power plant and they were that we had RFID tags on equipment. So they would.
Well, I'll give I'll give a last story here. And it's very similar to that. You know, working in these places, operators have we do what's called operator rounds where they walk around with some kind of handheld. It could be a piece of paper or whatever. In this particular area, it was a it was a power plant and they were that we had RFID tags on equipment. So they would.
The operator would walk up and they would scan the RFID tag and it would pull up the operator round portion and it would be a valve or a gauge or something. And they would, you know, put in readings, you know, temperature, pressure, you know, vibration. They put hands on it. They talk about if there's anything, any oil leaking or if it's dirty or whatever the thing is for that.
The operator would walk up and they would scan the RFID tag and it would pull up the operator round portion and it would be a valve or a gauge or something. And they would, you know, put in readings, you know, temperature, pressure, you know, vibration. They put hands on it. They talk about if there's anything, any oil leaking or if it's dirty or whatever the thing is for that.
And it could be, you know, 10 steps long or it could be 100 steps and it could take them all around the facility. And we started noticing as we were doing reviews of the operator round and the data, it just, the data didn't make sense. And we were trying to get better information to do tuning and all that kind of stuff. And it just didn't make sense.
And it could be, you know, 10 steps long or it could be 100 steps and it could take them all around the facility. And we started noticing as we were doing reviews of the operator round and the data, it just, the data didn't make sense. And we were trying to get better information to do tuning and all that kind of stuff. And it just didn't make sense.
And sometimes the numbers were like way off, like temperature would be 9 million. Like, yeah, I don't think so. So is that an error? Is that an operator error? Are they entering it wrong? Is a key sticking? Like what is going on there? So as I'm walking it and I'm doing this.
And sometimes the numbers were like way off, like temperature would be 9 million. Like, yeah, I don't think so. So is that an error? Is that an operator error? Are they entering it wrong? Is a key sticking? Like what is going on there? So as I'm walking it and I'm doing this.
And so I've got this guy and we're doing around and we're walking and he's scanning and we're going through it and everything looks fine. And then at lunchtime, we go in the break room where all the operators hang out and they're in there playing dominoes. And as they're playing dominoes, one of the guys, he's like, oh, hang on.
And so I've got this guy and we're doing around and we're walking and he's scanning and we're going through it and everything looks fine. And then at lunchtime, we go in the break room where all the operators hang out and they're in there playing dominoes. And as they're playing dominoes, one of the guys, he's like, oh, hang on.
And he leans back in his chair and right behind him, he's got the handheld in his hand. He leans back in his chair and he scans an RFID tag that's sitting on the wall. And he does something on his thing and he goes back to playing dominoes. And I was like, what did you just do? He's like, Oh man, I got to do a round right now, but I'm in the middle of the game.
And he leans back in his chair and right behind him, he's got the handheld in his hand. He leans back in his chair and he scans an RFID tag that's sitting on the wall. And he does something on his thing and he goes back to playing dominoes. And I was like, what did you just do? He's like, Oh man, I got to do a round right now, but I'm in the middle of the game.
I'm like, okay, what'd you just scan? He goes, oh, I made a copy of all the RFID tags that are out in the field. And I just scan them right here and I can do it right here from the lunchroom. So they had gone out of their way, gotten blank RFID tags, went through all the rounds and duplicated every RFID tag. And he knew how long it took him to get from one to the next.
I'm like, okay, what'd you just scan? He goes, oh, I made a copy of all the RFID tags that are out in the field. And I just scan them right here and I can do it right here from the lunchroom. So they had gone out of their way, gotten blank RFID tags, went through all the rounds and duplicated every RFID tag. And he knew how long it took him to get from one to the next.
And he was just randomly scanning them and putting in fake data.
And he was just randomly scanning them and putting in fake data.
i mean so so it goes back to your point of sometimes you don't need the rfid tag and the fancy thing to make it more efficient and know exactly when they're scanning sometimes you just need somebody to watch over and say hey that's not a good idea and those things had been there for a long time and nobody noticed nobody saw saw somebody it took me five minutes like i was in there for half a day and
i mean so so it goes back to your point of sometimes you don't need the rfid tag and the fancy thing to make it more efficient and know exactly when they're scanning sometimes you just need somebody to watch over and say hey that's not a good idea and those things had been there for a long time and nobody noticed nobody saw saw somebody it took me five minutes like i was in there for half a day and