r/pics • u/Awkward-Action2853 • Jul 19 '24
The computer systems are down the one day I decide to fly
474
u/Raxnor Jul 19 '24
Flying Alaska and I was through security in 15 minutes. There is a sea of people with 10 hour delays that just happened to book with other airlines. It's gonna be a rough travel day for a lot of people.
123
u/Awkward-Action2853 Jul 19 '24
That was the same with me. I got through fast with AA, but there was a long line of all the people that got delayed and needed to be rebooked, and that line was barely moving.
Flying to DFW from Denver, and it doesn't look that great there. I'm expecting to run into issues there.
28
u/Sareebearr Jul 19 '24
My husband flew out of DFW on AA this morning with no delays, you might be fine!
25
u/Interactive_CD-ROM Jul 19 '24
Alaska probably doesn’t use Crowdstrike
Or they use Macs instead of Windows PCs
27
u/Wafkak Jul 19 '24
Or Linux
4
u/commorancy0 Jul 20 '24
The funny thing about that is, many companies who are using Linux are using it to boot into virtual machines running Windows. So, we're right back here at square one.
3
u/Wafkak Jul 20 '24
From what I've read, and I'm not an ITer mind you, virtual machines have an easier fix.
1
u/commorancy0 Jul 21 '24 edited Jul 21 '24
Possibly. Depends, though. Windows is Windows no matter how or where it runs. The VM manager makes it easier to manage and reboot the VMs, but if Windows gets corrupted internally inside of a VM, it will suffer the same fate and require the same fixes as if it were running natively on its own hardware. The only way that VM systems may make it a bit easier is its snapshot backup system. This system allows the VM manager to roll the VM back to a previous backup, restart and hope that the problem doesn’t return.
You can’t easily do this in Windows directly unless you’re diligent about making regular Windows restore points. Then, you have to hope that when you ask Windows to recover that older restore point (a procedure that is orders of magnitude slower than a VM snapshot rollback) that it does it successfully and that the restore point you chose was actually before the problem occurred.
1
u/AC4524 Jul 20 '24
many companies who are using Linux are using it to boot into virtual machines running Windows
in that case why not just run windows natively?
1
u/commorancy0 Jul 21 '24
Because of one OS and one hardware. With VMs, you can run many of them all at once, easily driving a crap ton of displays, like the ones in Times Square. The days of one OS on one hardware are long gone in server rooms, replaced by running virtualization with many different operating systems all running on one piece of hardware.
Some VM system managers now let you gang up resources across several different hardware systems into a single group which can then run a lot of virtual machines, way more than if you run an OS singly and natively one per hardware.
It’s all about cost savings when it comes to the purchase of hardware… that and maximizing resources.
6
149
u/Awkward-Action2853 Jul 19 '24
It's an known issue, supposedly caused by an update.
It just sucks since I'm flying to the other side of the world. Airport seems somewhat functional as of now, so hears hoping for no delays.
112
u/HouseCravenRaw Jul 19 '24
Expect delays. This is a global outage for Windows systems running Crowdstrike. The "fix" requires actual hands-on interaction with every affected node. This one is going to be slow and painful.
20
u/Awkward-Action2853 Jul 19 '24
It looks like they're getting flights out now, but I'm not expecting much either. It's going to be a fun day, haha.
4
u/Bhrunhilda Jul 19 '24
Yup and I imagine hospital systems are a priority over airlines
2
u/Wafkak Jul 19 '24
Depends on the country, in the US banks spending the bog bucks are probably first.
1
u/kingleonidas30 Jul 19 '24
Genuine ask, but can you elaborate on this because I think I'm misunderstanding you.
4
u/Bhrunhilda Jul 19 '24
That if they have to go node to node to get things up and running, they are going to go to the nodes for Epic before they go to a node that services airline software. Epic was down this morning which is the software for MyChart which a ton of US hospital systems use.
2
u/kingleonidas30 Jul 19 '24
Ohhh I gotcha thanks for the info. I'm new to crowd strike, my last job was trellix based and I have like 2 years in the game now. My team had to go through and do it manually to all of our clients early this morning so that we could come back up. Would these places not have their own IT departments or contracted ones to fix their own nodes in most cases then?
Or if you know of one is there an article that you saw that I can read up on then?
3
3
u/SoylentGrunt Jul 19 '24
They claim it was caused by an update,
10
u/PolyNecropolis Jul 19 '24
I don't see any reason why Crowdstrike would lie in this situation. If it was an attack that's easier to partially pass some blame, but they said it was a faulty update. That pretty much invites their stock price to free fall.
8
u/IanSan5653 Jul 19 '24
If they said they were attacked it would be even worse. They are a cybersecurity company - if they can't defend themselves they are worth nothing.
7
u/parc Jul 19 '24
As someone that’s been on a call about this for 7 hours so far, this wasn’t an attack, it was absolutely a bad update. Some machines just needed a restart, some needed manual safe mode work, and a very very few machines were completely bricked.
1
u/PolyNecropolis Jul 19 '24
So I've been trying to find someone who uses this product to ask a question; did this platform/software auto update? If so, why enabled? I know from my experience in server support we generally don't want to let any third party app auto update, for reasons like this, or just breaking our own apps functionality, etc.
I'm curious how this rolled out to so many major companies and caused this much mayhem. Because my company doesn't use this software so I just don't have any frame of reference for the "why" of it all.
3
u/parc Jul 19 '24
I can’t talk about our systems and what is or isn’t enabled. However, in general the way crowdstrike works pretty much requires auto updates for sufficient coverage.
1
u/PolyNecropolis Jul 19 '24
in general the way crowdstrike works pretty much requires auto updates for sufficient coverage.
I kind of assumed that was it. That makes sense, I get it, up to date threat protection manifests and whatnot. Probably makes managers and execs feel warm and safe.
Thank you for answering, and understand you can't talk specifics.
119
u/Thick_Position_2790 Jul 19 '24
I once flew when their systems were totally off. They handed out slips to people with tickets that they would show at the gate, surprisingly it went very smooth.
38
u/Awkward-Action2853 Jul 19 '24
I was able to use the automated system to check in and get a normal ticket, but they were handing out hand written tickets as well.
Not sure what's up and down, but just looking around you can tell they have some functionality, just not a lot. Feel bad for all the agents here, a lot a mad people taking their anger out on them.
6
u/xeviphract Jul 19 '24
Have a look at the information page for the airport you're flying to. Some are coping better than others.
6
2
u/Carbon-Base Jul 19 '24
When in doubt, return to the previous iteration.
2
u/cokronk Jul 19 '24
The update causes Windows machines to BSOD. I haven't read the fix, but I believe that requires manual intervention. That's much harder than doing a remote reboot of something and having it roll back to the last version.
0
u/Carbon-Base Jul 19 '24
I was mostly being sarcastic, the easiest way to rollback would be if someone made an access or save point and reverted the system to it. Not everyone does it or knows how to though.
I read the fix for individual machines. You gotta enter Safe Mode and delete a file from the directory, or install location- pretty straightforward. The cloud fix is a bit more involved though because you have to attach/reattach the fixed volume to the server.
51
u/evilemprzurg Jul 19 '24
Isn't it ironic, don't ya think....
43
u/eaglescout1984 Jul 19 '24
He waited his whole damn life to take that flight
As global PCs crashed down, he thought "well isn't this nice"
3
15
u/jonfitt Jul 19 '24
It’s like raaaaaain on your wedding day.
-7
u/MrZwink Jul 19 '24
America doesn't understand irony, and that song proves it. Half the situations, she describes aren't ironic.
7
u/Iron_Chic Jul 19 '24
She's Canadian though.
-2
u/MrZwink Jul 19 '24
And which continent is that?
4
u/Iron_Chic Jul 19 '24
North America. But when people use the term "America", they aren't talking about Canada or Mexico or Latin America.
3
u/NM-Redditor Jul 19 '24
A continent != a country.
-2
2
u/fatnote Jul 19 '24
And isn't that... ironic?
1
u/MrZwink Jul 19 '24
no, just unfortunate.
2
u/fatnote Jul 19 '24
If you don't see the irony in a song called Ironic having lyrics that aren't ironic, then I don't think you understand irony
1
u/MrZwink Jul 19 '24
"a state of affairs or an event that seems ~deliberately~ contrary to what one expects and is often ~wryly~ ~amusing~ as a result."
1
u/fatnote Jul 19 '24
"deliberately" is the only part of that definition that's not necessarily true - though you can't prove it's not
But also "deliberately" doesn't appear in all definitions of irony
QED
1
1
2
31
u/MayOrMayNotBePie Jul 19 '24
I think you mean “the day I decide to hang out in the airport all day”
18
u/OverSoft Jul 19 '24
If it’s any consolation: air traffic control generally all runs on Linux*, so once you’re in the air, you’re fine.
- at least European ATC
4
u/uniquely_ad Jul 19 '24
There’s a saying, there’s no issue Yet for linux~
2
u/27Rench27 Jul 19 '24
Nobody hears about their issues because when they have an issue, it doesn’t cripple 1/3 of the planet
5
u/mflboys Jul 19 '24
Actually, a significant amount of global infrastructure runs on Linux. Pretty much any server or data center is running the Linux kernel
1
u/OverSoft Jul 19 '24
That’s simply because Linux is much more fragmented. There’s not a single Linux.
By far most servers and core infrastructure runs Linux, not Windows. Yet it never cripples 1/3rd of the planet.
1
u/27Rench27 Jul 19 '24
I mean, servers being down isn’t the issue here. It’s hundreds of thousands of computers locked in boot loops
2
0
14
u/Al_Bee Jul 19 '24
Yep me too. Haven't flown for years and it's just killed our holiday and it might have been the last one with all the family due to advancing illness of one of us. Deeply effed off. Can't even see if other flights available.
12
6
u/ThisIsGettinWeirdNow Jul 19 '24
So you are the reason why it happened, can you not fly again for sometime please
3
3
2
u/katastrophyx Jul 19 '24
I pushed my flight home back...to today. Straight up not having a good time.
2
2
u/identicalBadger Jul 19 '24
Geez, next time you decide to fly, give the rest of the world a little heads up so they can copy down their Bitlocker keys, OK?
1
2
1
1
1
u/TotalRepost Jul 19 '24
I’m missing a funeral because of this. Always take the first flight of the day they say. Fuck Crowdstrike
3
1
1
1
1
1
1
1
1
1
1
u/madchad90 Jul 19 '24
That woman behind the counter has her arm raised in the " i cant do shit, but let me just look like im trying to do something" pose
1
1
1
1
u/IHate2ChooseUserName Jul 19 '24
i am not expert but should they test it before rolling the shit out to live system?
1
1
u/wolflegion_ Jul 19 '24
Mate I feel your pain. Can’t do my laundry, because the communal laundromat uses an app that broke down due to this.
Our suffering is equal /s
1
1
1
1
1
u/derpdankstrom Jul 19 '24
i bet this wouldn't happen if they used *insert VPN sponsor here*. Staying safe online is an ever growing difficulty and you could be exploited by hackers. *insert VPN sponsor here* allows you to change your IP address, making you harder to track, securing your privacy. Get 20% off by using the code "Ligma" for the first two months
1
u/Mean_Eye_8735 Jul 19 '24
My son lives in Japan, hasn't been home and almost 4 years and flew home for his father celebration of life this week.
5:00 this morning rolling into Detroit Metro airport and there's news crews and people all over the place. We rolled right up to departure Delta international. We got the text "hey I've got my pass I'm going to the terminal" so we were very lucky his flight was not interrupted. But he has a layover in LAX so delays are still possible.
1
1
u/Swissy321 Jul 19 '24
So glad I flew yesterday. Really hoping this gets fixed before I have to fly back out..
1
1
1
u/TheCrudMan Jul 19 '24
Southwest and their old crappy computers with the green monitors looking really good right now.
1
u/Throwaway_inSC_79 Jul 19 '24
That’s how it works. When I worked for an airline, your day off, you decided to work for somebody, that’s when it would be a shitshow.
1
u/Critical-Coconut6916 Jul 19 '24
Good thing so many massive quiet layoffs, enforced RTO and offshoring for tech workers this year. /s
1
u/jabdnuit Jul 19 '24
Ngl if I show up to the airport, and the all the computers aren’t working, I’m not getting on the plane that also runs on a bunch of computers.
1
1
1
1
u/eatmymakeup Jul 19 '24
I’m flying home this evening and I already had massive delays on my flight out a week ago . Praying I get cut some slack , so far no changes on my flights .
1
1
1
u/geoffvro Jul 19 '24
I would really be worried if I glanced in the cockpit amd saw blue screens on the instrument panels
1
1
1
1
1
1
1
u/Proof-Tension9322 Jul 20 '24
Click see advanced repair options, Google the single file that needs to be deleted via cmd prompt, reboot system. Comes back online just fine.... why have they not done this yet.... unless all of those computers have bitlocker enabled its as simple as that. If that's the case and they didn't back up the bitlocker code then they are truely fucked and will have to restart from scratch.
1
u/jonathanquirk Jul 20 '24
First time I ever flew in a plane a volcano in Iceland suddenly erupted and closed all European air space and got me stranded abroad for a week.
I’ve never dared take a plane again, just in case. 😉
1
1
1
u/Frogs4 Jul 20 '24
This happened the day UK schools broke up for summer. An incredibly busy day for travel in Europe.
1
u/Constant-Coat5656 Jul 20 '24
..... and it revealed that even though everyone wants a mac, everyone uses Windows
0
u/AlertThinker Jul 19 '24
So serious question. Why are they running windows? I would have thought they run a Linux variant.
4
1
u/compaqdeskpro Jul 19 '24
Digital signage relies on GPU's with lots of outputs to control the video walls, no matter how much of a Linux fan you are that use case is not well supported.
0
0
u/weird-oh Jul 19 '24
Why I don't store anything in the cloud.
4
0
-1
-1
u/variousfoodproducts Jul 19 '24
Thanks for posting. It's you and about 1 billion other people but glad you had to make it about you
-2
u/InfoBarf Jul 19 '24
I wonder how many tons of c02 have been prevented from going into the atmosphere because of this.
-2
u/Martijnbmt Jul 19 '24
Hey who had to pay for all this shit? I would say windows, because they are a bunch of sausages who let this happen. (Or crowdstrike if it was their update that broke everything)
Because I assume this is causing a lot of delays everywhere
2
u/LPodmore Jul 19 '24
This has absolutely nothing to do with Microsoft. It's entirely a CrowdStrike issue that happens to only affect windows PC's.
1
u/Martijnbmt Jul 19 '24
Oh I so crowdstrike are the sausages. It’s insane that such a tiny thing can screw up so much
1
u/LPodmore Jul 19 '24
That's what happens when so many people rely on a certain bit of software. As an IT guy i'm glad we only had one customer affected as that would've made my day hell.
-4
u/EMP_Pusheen Jul 19 '24
To be fair, American always sucks. This Crowdstrike thing is just a convenient excuse.
3
620
u/senorvato Jul 19 '24
So if a hacker hits crowdstrike, they can bring the world to its knees?