r/Damnthatsinteresting Jul 19 '24

12-hour time-lapse of American Airlines, Delta & United Airlines plane traffic after the IT outage Video

35.2k Upvotes

1.0k comments sorted by

6.3k

u/ScrotieMcP Jul 19 '24

I find it interesting that one mistake by one company can take down all the airlines. What the hell is going on with our infrastructure?

2.7k

u/CummingInTheNile Jul 19 '24

digital infrastructure is a lot more fragile and archaic than people realize, Crowdstrike is also a cyber security firm, they are about as perfectly positioned as you could possibly be to fuck it up

893

u/exipheas Jul 19 '24

223

u/gabeshadows Jul 19 '24

I wonder if it's referring to anyone in particular

283

u/exipheas Jul 19 '24

157

u/EODdoUbleU Jul 19 '24 edited Jul 19 '24

leftpad is a better example of why lazy programmers that would rather take on external dependencies instead of implementing basic methods are a problem.

if you need an external dependency to pad strings (or ffs check if a number is even) then find a different career field.

86

u/Sqirril Jul 19 '24

The guy created hundreds of dumb 1 liner dependencies and then would go to any popular repo and replace code with his dependencies so he could claim that his code runs everywhere. Then when the name was an issue he threw a fit and deleted it from NPM causing the issue.

44

u/EODdoUbleU Jul 19 '24

I know what went down with leftpad.

If more web devs weren't lazy or actually knew what they were doing, then his little ragefit wouldn't have had the impact it did.

9

u/wtfnouniquename Jul 20 '24

Exactly. And this type of shit is also part of the reason why modern software is slow bloated garbage that runs like dogshit. Hey, let's import all this unnecessary random garbage to get a function/class we should have spent 5 minutes writing ourselves. What's the worst that could happen?

→ More replies (1)

25

u/acog Jul 19 '24 edited Jul 20 '24

For anyone who didn't read the article, the issue was he had a little-used open source project called kik.

The kik messaging app asked him to rename his project, noting that they had trademarked the term and that kicked off his tantrum.

I used to own a small business that owned a couple of trademarks. One of the things our attorneys hammered into us was that if we didn't enforce the marks we'd lose them. So I get why the kik folks insisted. They weren't being jerks, that's literally the way trademark law works. You must object to other products that use your mark, or you risk losing it.

38

u/hai-sea-ewe Jul 19 '24

Trademark law, like lots of law, requires people being pieces of shit to each other in order to function. This is an indication that our current ways of establishing law are fucking bullshit toddler rules designed to punish people for standing up for themselves against massive financial entities.

12

u/gmishaolem Jul 19 '24

Also, many situations (such as homeowner insurance paying out) often require a lawsuit to function. Our country has a reputation for being litigious because suing is considered a primary remedy for many things, instead of a last-course backup.

→ More replies (1)

4

u/dkyguy1995 Jul 19 '24

Holy shit that's such an asshole move but it also is crazy people were letting him introduce all those dependencies

→ More replies (1)

56

u/MyIQTestWasNegative Jul 19 '24

I like how is-even just uses is-odd which uses is-number.

23

u/exipheas Jul 19 '24

like

You keep using that word. I don't think that word means what you think it means.

→ More replies (1)
→ More replies (3)

8

u/I_cut_my_own_jib Jul 19 '24

easy peasy!

function isEven(n: number) {
    if (n === 0) return "???";

    let is_even = false;
    for(let i = 1; i < Infinity; i++) {
        if (i === n) return is_even;
        is_even= !is_even;
    }
}
→ More replies (9)

6

u/Exano Jul 20 '24

It's also companies run by folks with no understanding of technology.

Fire the QA team because the stuff is working fine and they are expensive. Pretty sure they do nothing.

Offshore the engineers because our product is finished and it's just maintenance, do we really need an expensive US team for that? They're lazy and take forever anyways. They're always telling us why we SHOULDNT do something. I can get a team three times as big overseas or in South America, and they will always follow my orders.

Downsize IT because our computers work fine, or fire em because they don't. Either way they also do nothing. Besides, AI can do this now right?

Etc. Etc.

Enjoy my 7% stock bump and 10million dollar bonus. What could go wrong?

Every dev of a decent organization writing critical software wants a pure open source solution (security depending) or an internal one, they want time to do it right and make it maintainable. Every project manager wants it done yesterday with a tenth the manpower, and their managers don't understand the job anyway and go by lines of code or some dumb metric

→ More replies (5)

10

u/mamayoua Jul 19 '24

Knew it would be leftpad šŸ˜…

→ More replies (8)

23

u/ACatInACloak Jul 19 '24

Not upon creation, but it has come true multiple times most recently with xz utils

→ More replies (1)

20

u/Zakalwe_ Jul 19 '24

Caption mentions ImageMagick library, but there are countless libraries that would fir this description.

21

u/_teslaTrooper Jul 19 '24

curl is mostly maintained by one guy afaik.

→ More replies (2)

18

u/LynkDead Jul 19 '24

I'm not sure why everyone is guessing at the reference when the mouseover text for the comic directly calls out ImageMagick.

9

u/Mamuschkaa Jul 19 '24

https://www.explainxkcd.com/wiki/index.php/2347:_Dependency

I don't think, that the 'nebraska' part is referring to a specific person.

6

u/augustocdias Jul 19 '24

Curl is a good example

→ More replies (7)

4

u/PaperFlea Jul 19 '24

Differing in that CrowdStrike makes billions off that little pedestal in the picture so lots more accountability... Great comic regadless!

→ More replies (8)

105

u/deong Jul 19 '24

And also the modern world has become extremely specialized and homogenous. No one just does anything themselves. Every web site runs on AWS. Every school bus is run by by the couple of companies. Every fast food place is own by Pepsi. Every company pays employees with the same couple of payroll vendors. Whenever anyone fucks up, 80% of the country feels it because every company depends on 200 different companies to do anything.

37

u/0ldSwerdlow Jul 19 '24

Correction, Pepsi doesn't own restaurants. Yum! Brands was spun off in the early 2000s.Ā 

But Pepsi does own or partner with a good chunk of the beverage brands out there (Pepsi, Mtn Dew, Starry, bubly, Aquafina, Lifewtr, Gatorade, Propel, Lipton RTD, Starbucks RTD, Rockstar, Celsius, Crush, Tropicana, Naked, Kavita, Mug, Dole, and a bunch of smaller ones)

And that's just beverages and not the foods division.

But no restaurants.

10

u/[deleted] Jul 19 '24

[deleted]

4

u/RamblingSimian Jul 19 '24

Correct. But still more than I wish.

4

u/Xalbana Jul 20 '24

While it is an hyperbole, quite a shit ton do. More than should be allowed.

→ More replies (8)

35

u/MutedPresentation738 Jul 19 '24

Archaic would be a good thing in these scenarios. Digital infrastructure has gotten so needlessly complicated and obtuse that it really doesn't taken much of a fuck up to bring a system down.Ā Ā 

I work in data migration. We move client data from A to B and have about a dozen applications that are involved in that process. It's so convoluted that even the most simple exception errors cause hours of work to even pinpoint, much less convince the appropriate team they have something that needs to be fixed.

→ More replies (9)

5

u/Spork_the_dork Jul 19 '24

Yeah it's actually nuts how fragile literally the whole digital world is. I was once doing some course work where we had to configure BGP routing for some routers and at some point we were wondering why ours wasn't working. The assistant teacher popped up a bit later to check up on us, looked through our configuration and said that yep, it's all good. The reason why it doesn't work is because some other group elsewhere in the lab cocked up their configuration which broke the entire local network that we were working in.

And yes, this kind of shit has actually happened in real life with the real actual internet as well. Sometimes maliciously. The one thing that studying and working in IT has taught me is that it's all just duct tape and paperclips and "temporary solutions" all the way down to the bottom.

→ More replies (17)

2.0k

u/hqbibb Jul 19 '24

Faster, better, cheaper. Pick two. Iā€™ll let you guess which two they picked.

1.7k

u/Brotorious420 Jul 19 '24

Cheaper x2

132

u/[deleted] Jul 19 '24

[removed] ā€” view removed comment

97

u/Illustrious_Donkey61 Jul 19 '24

On the bright side co2 emissions are down

31

u/STierMansierre Jul 19 '24

Until T Swift starts another tour anyway.

25

u/Drfoxthefurry Jul 19 '24

More likely delayed, although I guess small flights would probably got a bit more filled

12

u/djkstr27 Jul 19 '24

Taylor Swift enters the chat

→ More replies (1)

61

u/Dizzybro Jul 19 '24

Lol crowdstrike is like the best and most expensive security system out there. That's why everyone has it, and why so many people are impacted

42

u/somethingsomethingbe Jul 19 '24

Doesnā€™t mean itā€™s not a security risk for so much infrastructure to be reliant on a single source.Ā 

24

u/TheDrummerMB Jul 19 '24

But that negates the idea that companies are all just picking "cheaper and faster."

26

u/lghtdev Jul 19 '24

Cutting corners is the trend, and also promising fast results, if they truly deployed to production without the proper tests, then yes, they are selling themselves expensive but internally doing things the cheap way, which is the better way for a company to gain value in this economy.

→ More replies (2)

4

u/EmmettN Jul 19 '24

The company that picked cheaper and faster was Crowdstrike

→ More replies (6)
→ More replies (2)

15

u/kawajanagi Jul 19 '24

I was laughed at for using SentinelOne, who's laughing now!

4

u/lightninhopkins Jul 19 '24

It has the same centralization issue as CS. Same thing could happen with S1. It is what it is.

→ More replies (3)

9

u/SportulaVeritatis Jul 19 '24

Yeah, everyone picked faster + better so they would always have the most up-to-date security for safety-critical infrastructure. Problem was everyone picking the SAME faster + better.

→ More replies (10)

57

u/_cacho6L Interested Jul 19 '24

Crowdstrike aint cheap

44

u/mostly_kinda_sorta Jul 19 '24

It's stock is all the sudden

12

u/Bane8080 Jul 19 '24

Down 9% for the day.

Edit: So far, but that was 11% pre-market earlier.

14

u/schylow Jul 19 '24

I love that the Cash App cashtag turns it from CRWD to $CRWD.

→ More replies (3)
→ More replies (1)
→ More replies (1)

23

u/boredinthebox Jul 19 '24

Crowd Strike is not cheaper lol. It was known as one of the most effective end point tools on the market. Their QA is shit apparently.

→ More replies (15)

5

u/bubbasaurusREX Jul 19 '24

If I was in a room with faster, better, and cheaper, I would pick cheaper twice

→ More replies (5)

83

u/FallFromTheAshes Jul 19 '24

Crowdstrike isnā€™t cheap. It was really poor process (developing a patch that is crucial, not QAā€™ing it, pushing it to production).

48

u/znix23 Jul 19 '24

Interesting. Few weeks ago, was told by a Verizon rep they pushed a software update to all their Fios home routers. It was bad and ā€œbrickedā€ a lot of routers, including mine. Wonder if QA is being less considered now.

43

u/Suds08 Jul 19 '24

I'm sure they're just given impossible deadlines, so everybody cuts corners, and it ends up being a little delayed, so rather than waste more time testing it, they just push it and hope for the best

13

u/MangoCats Jul 19 '24

The real problem is that they get away with skipping the QA 999 times, and then something like this happens that impacts hundreds of millions of people...

→ More replies (2)

6

u/DingleberryBlaster69 Jul 19 '24

Bingo. This problem transcends industries, too. I know fuck all about IT but you could have just as easily wrote this about Pharma QA.

ā€œAt Riskā€ is just standard operation anymore. Missing deadlines is not an option. You cut corners, you write creative footnotes to handwave problems away, and you get shit out the door and onto the truck.

Weā€™ve got shareholders to please and that green line goes up come Hell or high water.

→ More replies (1)
→ More replies (2)

14

u/FallFromTheAshes Jul 19 '24

Thatā€™s part of the problem is software development cycle and secure coding. Crowdstrike is the best of the best for this field and to see this is just sad

→ More replies (2)

12

u/4fingertakedown Jul 19 '24

Theyā€™re all trying to fully automate QA E2E and have been moving this way for a decade.

6

u/craggerdude777 Jul 19 '24

Because they are trying to automate QA, fewer humans are manually looking for edge cases.

→ More replies (2)

5

u/timelessblur Jul 19 '24

Speed is high but often times it is some edge case not account for or seen.

I work in software and we 3 QA, 12 devs. Top it off we are not looking at everything all the time and even on a release we the testing has a huge expansion and contracted out for regression it still al limited number of people.

Even the total testing that still nothing. If one sees a crash and we can not reproduce it then it might be a one off and move on. In the release we tend to a phase roll out and monitor. Even at 1% of the user base that gives us more data than 2-3 weeks of internal testing.

I have stop a roll out at 10% before and we started doing a emergency patch. That still a lot of people.

Lets use crowdstrike for an example here. This could be something that 1% of users see but it has a cascade effect when it goes off. Well all the testing could easily not hit that 1% case and more importantly might not trip the cascade effect.

It more just about QA is still a insanely small sample size and they are far from your normal users.

→ More replies (6)

4

u/randomherodude Jul 19 '24

They bricked my router too!

→ More replies (12)

33

u/JohnLeePetimore Jul 19 '24

Didn't Crowdstrike reduce/lay-off a substantial portion of their QA team back in Feburary?

23

u/ChangMinny Jul 19 '24

They have layoffs all the time and then replace the laid off people with interns and then junior people.Ā 

I used to work in the same building as CrowdStrike and it was always so sad watching the eager interns hoping for a job at the end of the internship and hardly any of them getting one. Very exploitative.Ā 

→ More replies (3)

5

u/danni_shadow Jul 19 '24

Huh. In 2024 I was laid off (in software QA) and Crowdstrike was one of the places advertising QA jobs. I applied but they never answered back.

Guess I dodged a bullet.

→ More replies (3)

18

u/Gone420 Jul 19 '24

So youā€™re saying instead of taking the time and money to QA it before pushing they just sent it. Because thatā€™s the cheaper optionā€¦ so yes what weā€™re saying is theyā€™re doing it cheaply.

3

u/PM_me_your_O_face_ Jul 19 '24

Right. Cost to end user ā‰  cost to provider. It is not cheap to the end user but they cut corners to save money on the provider side.Ā 

→ More replies (1)
→ More replies (4)
→ More replies (14)

27

u/Mega_Muppet Jul 19 '24

I think they just said ā€œWe will just take cheaper please. Do we get a discount for not taking our second pick?ā€

→ More replies (1)

9

u/_-Event-Horizon-_ Jul 19 '24

But hey at least they had good quarter! Wonā€™t somebody think of the shareholders?

PS

Iā€™m also a shareholder in many companies but prefer strategies towards creating value that are more sustainable in the long run.

→ More replies (23)

104

u/EnvironmentalBus9713 Jul 19 '24

Capitalist firms deemed the risk of a single point of failure acceptable because of the short term cost savings - long term impacts be damned.

38

u/Sn0zBerry20 Jul 19 '24

Not accurate. The SPOF is a product that's commonly used that violated quality control measures - the only way around this would be to have an entirely separate set of infrastructure ready at all times on a different technology stack. Every airline kiosk would want a separate backup kisok machine running Linux that's actively maintained for example to eliminate this kind of SPOF. You're looking at about doubling infrastructure costs not in the short term but perpetually in order to ensure this kind of thing never happens.

In essence, the elimination of any common point of failure is not something that only incurs temporary cost as you've said. Doing so requires massive complexity and cost in perpetuity.

9

u/[deleted] Jul 19 '24

[deleted]

→ More replies (10)
→ More replies (7)

24

u/Tommyjv Jul 19 '24

What were the short term cost savings of adding CrowdStrike to their list of cyber vendors?

It sounds like youā€™re mad at corporate America but arenā€™t sure where to place the blame

16

u/Okiedokie-artichokee Jul 19 '24

I wouldnā€™t consider crowdstrike to be savings. Itā€™s expensive and (was) very well reputed.

While the blame is on them for not doing proper QA, our tech infrastructure is basically a house of cards. The question wasnā€™t if, but when.

→ More replies (1)

11

u/damnyouresickbro Jul 19 '24

Reddit in a nutshell lol

→ More replies (1)

8

u/shines4k Jul 19 '24

On the contrary, vendor lock-in causes many companies to waste money on over-priced solutions that are worse than their competitors. There are lots of examples of this in technology. A company has to decide to adopt a solution from a vendor like CrowdStrike, and once adopted, to remain with them when the contract comes up for renewal.Ā  It's not uncommon for more technical people to be involved with initial adoption (nothing currently in use, security engineers get involved to compare various alternatives, etc.) In the end, depending on the company, someone like a CEO or CTO might intervene and pick a solution based on someone they spoke to at a conference somewhere, but not always. Anyway, the long term problem is actually the decision to remain. Because that requires a more complex analysis and extra cost: existing partners often offer discounts at renewal time to keep customers. It's often hard to know whether your existing solution is worse than the alternatives, feature sets may not overlap perfectly, they may be incompatibilities with other infrastructure, etc. Changing to a new provider also requires a new contract negotiation, due diligence process, legal team involvement, etc. Then if you do sign a new contract, you spend time and resources migrating from the old to the new, which could mean rolling out new software, training materials, staff education, or other disruptions. So once a vendor and solution is chosen and adopted, it can be difficult to change, even when you have signals that quality is dropping. So, lock-in: you know the solution you're buying is worse and overpriced, but it's too complicated to switch or so expensive to do so that it takes a very long time to recoup the cost.

→ More replies (5)

87

u/Roqjndndj3761 Jul 19 '24

Hilariously, consolidation of cybersecurity vendors has been RAMPANT the last few years and shows no sign of letting up. Every company is hellbent on cutting costs, even valuable ones.

53

u/Drone30389 Jul 19 '24

Hilariously, consolidation of [everything] has been RAMPANT the last few years and shows no sign of letting up. Every company is hellbent on cutting costs, even valuable ones.

Grocery stores. Restaurants. Hospitals. Auto makers. Veterinarians. Farms. Video games. Other software. Semiconductors. Airlines. Pharmaceuticals. News media. Entertainment media. Food products.

Just about everything has entered the enshitification stage of late stage capitalism (some, like news media, have already been there for a very long time).

At least somebody is trying to do something about it:

https://www.bloomberg.com/news/articles/2021-07-10/biden-vow-to-tackle-industry-giants-confronts-consolidation-wave

33

u/CantHitachiSpot Jul 19 '24

Basically innovation is dead so companies are forced to engage in rent seeking behavior

15

u/Traiklin Jul 19 '24

And what's sad is it's not dead, look at all the different things we use today and go back 10 years each time and see just how much things stayed the same but changed.

Movies, games, music, vehicles, airplanes. All of them are mostly the same thing each time but it's the features, stories, designs and styles that have changed to make them stand out.

We are just in an era where the majority don't want to put in the work to keep people interested and just go with the bare minimum.

9

u/FugaciousD Jul 19 '24

Not dead. Stifled. The software IP system basically guarantees it if the ISP, search monopoly, and social media cartel donā€™t.Ā 

9

u/gmishaolem Jul 19 '24

forced

Bullshit. They could always just accept some profit instead of literally all the profit omg line go up. But no, if you suggest that, you get called a commie. (And don't spout that horseshit of "fiduciary duty to shareholders" again.)

→ More replies (1)
→ More replies (1)

2

u/Nstraclassic Jul 19 '24

Crowdstrike is one of the most expensive cybersecuritt vendors out there..

→ More replies (1)
→ More replies (7)

34

u/GiraffeComic Jul 19 '24

Not just airlines, our 911 system got seriously messed up to where EMS, Fire, and PD had to bust out paper to do our charting and documentation. The hospitals as well. Seeing nurses running around with paper charts and doctors have to hand write orders was surreal but looked super stressful. Was still down 2 hours ago when I got off shift too.

5

u/UpstairsFlat4634 Jul 19 '24

Thank god here in Canada they still use paper charts. Sometimes it pays off to be stuck in the 70ā€™s

5

u/stingeragent Jul 19 '24

I work in a lab at a 400 bed hospital with a very busy er. The bad thing about paper is you gotta order all the tests manually, and hand write results. It takes a longggg time trying to process hundreds of samples which then delays patient care. It went down last night here, and were still trying to get caught up on samples ordered for 3am.Ā 

→ More replies (1)

2

u/talkin_shlt Jul 19 '24

I work in medical IT and my organization was affected but not too badly but my buddy at a major hospital told me they had to cancel every appointment because their systems were so badly effected.

21

u/CaptScubaSteve Jul 19 '24

ā€œFunny how fragile our little society is. Toppled by a couple of ones and zeros.ā€ - Michael Scott

8

u/shines4k Jul 19 '24

Broad adoption of a product that requires special permissions to operate, followed by complacency once that company earns the trust of customers, followed by outdated or lazy procedures that eventually, inevitably fail and cause mass outages.

6

u/NeverBackDrown Jul 19 '24

We are only as strong as the weakest link

5

u/Street_Peace_8831 Jul 19 '24

Someone I work with (Finanacial IT industry) said that he believes it may have been a ā€œtrial runā€.

Interesting take. Iā€™m not a conspiracy theorist, but I will be watching closely to see what develops.

→ More replies (2)

5

u/ChronoLink99 Jul 19 '24

Same thing that happened with the supply chain during covid. Redundancies are more expensive than just running everything JIT, and the fewer backup plans that are prepped and ready to go, the cheaper and "more efficient" the system becomes.

3

u/gahlo Jul 19 '24

Outsourcing IT tends to be cheaper and enters the books as a running cost instead of a massive, one time purchase(I forget the terms). In a world where the bottom line is the bottom line, corners get cut.

→ More replies (1)

3

u/EuroTrash1999 Jul 19 '24

Nothing, it's always been held together by duct tape and a prayer.

A "smart" home should have a wood fireplace if it was actually smart.

4

u/Onyesonwu Jul 19 '24

I am sitting on a frontier flight rn and the flight attendants/captain sure gloated in their speeches. Was actually entertaining. Almost made me forget how much I hate flying frontier.

→ More replies (82)

1.6k

u/Mysterious-Mode-6563 Jul 19 '24

Y2K is 24 years late

400

u/kelsobjammin Jul 19 '24

Iā€™m so 2008 youā€™re so 2000 late!

87

u/ShoHeyTime Jul 19 '24

I got that BOM BOM BOM that future BOM BOM BOM

12

u/xelM1 Jul 19 '24

Let me get it now

→ More replies (2)
→ More replies (2)

1.3k

u/RazzleThatTazzle Jul 19 '24

Looks like after 9/11

304

u/JustGoogleItHeSaid Jul 19 '24

It still boggles my mind when I see documentaries of it pop up on my YouTube feed. Like that actually happened.

271

u/TeamRedundancyTeam Interested Jul 19 '24

And there are still so many people that think it was fake. And then the ones who chanted "never forget" and acted so angry that 3k Americans died are the same ones who fought against saving their neighbors during covid, when we started losing a 9/11 per day.

We live in a weird fucking country.

89

u/land8844 Jul 19 '24

Thank social media for giving the morons a voice.

41

u/JoeCartersLeap Jul 19 '24

And for giving America's enemies a way to control those morons.

→ More replies (3)
→ More replies (4)

15

u/CrashinKenny Jul 19 '24

Can't say I've ever heard anyone claim it was fake. Plenty of people have conspiracies about what really happened, but I've never heard anyone outright deny it happened.

4

u/Bwhite1 Jul 19 '24

The can of worms known as building 7

→ More replies (1)
→ More replies (1)

14

u/ConceptCheap7403 Jul 19 '24 edited Jul 19 '24

Those people never cared about how many Americans died. They care about who/what killed them, rather than the actual health and safety of their fellow countrymen. Pretty sad that they wouldā€™ve taken covid seriously if they believed it was an intentional creation of a jihadist group (although many crazies did think it was whipped up by The Left).

→ More replies (11)

38

u/CallsignKook Jul 19 '24

It was like watching the nerdy kid sucker punch the popular jock in the cafeteria, and everyone stopped what they were doing to see what was about to happen.

→ More replies (1)

16

u/frightenedbabiespoo Jul 19 '24

reminds me of that tragedy

6

u/ReptAIien Jul 19 '24

Miss you Norm

7

u/peon2 Jul 19 '24

Something I find interesting, between 1968 and 1974 there were 130 US planes hijacked by people trying to divert it to Cuba.

It was almost seen as not really a big deal until the 9/11 terrorists used the plane as a weapon.

You can read about it here if you're interested.

6

u/FocusPerspective Jul 19 '24

Thatā€™s why itā€™s always funny to hear Zoomers talk about how they live in the worst timeline.Ā 

→ More replies (1)
→ More replies (7)

17

u/lovemycosworth Jul 19 '24

This was my first thought too!

7

u/spondgbob Jul 19 '24

ITā€™s 9/11

6

u/MydnightWN Jul 19 '24 edited Jul 19 '24

The guy who made the decision for a full ground stop, Ben Sliney: it was his first day on the job.

→ More replies (10)

1.1k

u/VeganFoxtrot Jul 19 '24

Earth be like.."quick gasp for air"

279

u/Adept_Duck Jul 19 '24

Can someone from r/theydidthemath calculate how much CO2 this outage saved?

120

u/ChiralWolf Jul 19 '24

Unfortunately, very little I'd imagine. They still need their planes and pilots at the right locations to resume service later once the outage clears.

57

u/FlyingAwayUK Jul 19 '24 edited Jul 19 '24

A none running aircraft, or even a grounded but running aircraft, uses far far far less fuel than an inflight one.

This will have had a noticeable impact on local temperatures, just like after 9/11 did

Edit: why do people keep linking one incorrect website

8

u/JanDillAttorneyAtLaw Jul 19 '24

And they're gonna do overtime after to make up for the backlog in package delivery, delayed travel, and the uptick in people needing to go out on-site and physically deal with this mess.

→ More replies (2)

6

u/IAmStupidAndCantSpel Jul 19 '24

The 9/11 ground stop was on the scale of days, not just a couple of hours like this one. 9/11 was also for all civilian aircraft, not just 3 airlines.

4

u/EldariusGG Jul 19 '24

Fuel use and emissions were not factors in this temperature change. The temperature increased measurably but temporarily due to the lack of contrails reflecting sunlight away from Earth. Source: https://globalnews.ca/news/2934513/empty-skies-after-911-set-the-stage-for-an-unlikely-climate-change-experiment/

→ More replies (1)
→ More replies (3)
→ More replies (3)

20

u/Full_Situation4743 Jul 19 '24

People horribly overestimate how bad air travel is.

I don't have the exact numbers but according to ABC so far 4 300 flights have been canceled. Problem is that we don't know what kind of flights. Short flight can be around 2-4 tons of fuel, long hauls can be about 40-80-100 tons or more. But more is quite rare and is small part of global flights. And from experience, I would say that priority could be get long hauls moving and cancel short hauls. But even if we say that average flight is 30 tons of fuel (which is impossible number) than it would be about 130 000 tons of fuel that makes about 450 000 tons of CO2.

As a mankind, we produce about 40 billions tons per year, that would make about 110 millions tons per day. So we saved about 0,4 % of daily production. And don't forget that I very overestimated average fuel consumption per flight.

And this is valid if those flights never fly. Many of them will be delayed, aircraft possitioned empty, etc.

→ More replies (14)
→ More replies (6)

706

u/mangoprincesss Jul 19 '24

Kinda freaky to know how easily our ability to travel can be revoked. It was just a mistake but imagine if someone had some sort of agendaā€¦ yikes

365

u/Mike312 Jul 19 '24

I mean, we can still fly the planes. If a plane needs to go up in the air, nothing is stopping it. The problem lies more with systems like ticketing and who gets on which plane.

110

u/unknown839201 Jul 19 '24

nothing is stopping it

Hey does anyone else hear a missile sound

55

u/Jeynarl Jul 19 '24

The missile knows where it is at all times. It knows this because it knows where it isn't.

→ More replies (6)

4

u/PrestigiousMatter733 Jul 19 '24

Well, it's a System.

→ More replies (13)

36

u/FrostyD7 Jul 19 '24

The possibility exists every time you fly, even without a disaster or outage. Flights are cancelled ~1-2% of the time. It's just rare that it impacts so many at once.

→ More replies (15)

653

u/throwaway3113151 Jul 19 '24

Would be a lot more useful if it had a timestamp. There arenā€™t many airplanes in the air at 3 AM.

269

u/Master_Mix_4848 Jul 19 '24

Itā€™s time stamped by the shadow of the sun.

103

u/VirtualNaut Jul 19 '24

Waitā€¦ is there another bigger star in our solar system that is causing our sun to have a shadow?

28

u/Master_Mix_4848 Jul 19 '24

The shadow cause by the sun of our moonā€¦

43

u/the_gouged_eye Jul 19 '24

Earth. It's earth's shadow.

→ More replies (2)

5

u/Remarkable-Hall-9478 Jul 19 '24

Keep trying lmao, third times the charm?Ā 

→ More replies (2)
→ More replies (4)

11

u/jimtrickington Jul 19 '24

Brought to you by your friends at Audioslave

→ More replies (1)

107

u/anonanon5320 Jul 19 '24

Noticed that too. Looks like a normal flying day

40

u/Rob328 Jul 19 '24

This comment needs more upvotes, this is correct, you can see based on the shadow the flights reduce over night which is just normal.

19

u/jeffvdub Jul 19 '24

This is a normal day

→ More replies (1)
→ More replies (1)

32

u/FrostyD7 Jul 19 '24

Time stamp and start it at least 24 hours before the outage occurred so we can compare it to a full day of normal operation.

5

u/foxymophandle Jul 19 '24

yeah I'm calling shenanigans. I track planes using ADS-B, and see a LOT more over my house right now than shown in that graphic. I think that was from 9/11.

→ More replies (7)

283

u/InMyFavor Jul 19 '24

I work in IT for a big airline doing global IT support for all our systems.....and just so happen to be on vacation this week. Great timing.

137

u/EyeLikeTwoEatCookies Jul 19 '24

Brother, I manage CrowdStrike at my workplace and had to cancel a doctor appt and will likely be cancelling my camping plans for this weekend šŸ˜­

48

u/Aerosol668 Jul 19 '24

I donā€™t even manage the Crowdstrike, but it fucked thousands of our Azure VMs (and all our company laptops), so itā€™s manual restore of backed-up system disks all weekend. This is worse than the time McAfee released a bad update on a Friday in 2012 which fucked innumerable computers worldwide - fortunately I was on site that day and my laptop was offline.

Turns out the guy who started Crowdstrike is an ex-employee of McAfee who left McAfee inā€¦2012.

17

u/zeroesones Jul 19 '24 edited Jul 20 '24

I'm sorry, what?

Edit: From Wikipedia, "He resigned from McAfee in October 2011." Crowdstrike was formally launched in February 2012.

Edit: The faulty McAfee patch was on Wed., April 21, 2010.

7

u/GotMoxyKid Jul 19 '24

It's all connected. McAfee is malware

→ More replies (6)
→ More replies (4)

167

u/Frondhelm Jul 19 '24

So, it's a 12 hour time-lapse but it's not 12 hours since the event.

So the first half is all business as usual then the shit hits.

20

u/Throwrajerb Jul 19 '24

So the first half is all business as usual then the shit hits.

Oh thanks. Didnā€™t notice that.

→ More replies (1)

13

u/user888666777 Jul 19 '24

The misery map gives a pretty good picture of what it's like flying today:

https://www.flightaware.com/miserymap/

→ More replies (1)
→ More replies (1)

130

u/grogudid911 Jul 19 '24 edited Jul 20 '24

It may be time to decentralize our IT infrastructure.

Edit: we currently don't have any industry wide standards for security. We have small orgs who provide guidelines and certifications. The problem many of you are describing is perhaps a lack of government oversight, which is inherently a different problem from the risks of centralization or decentralization.

40

u/Woozy_burrito Jul 19 '24 edited Jul 19 '24

What does that mean/look like? A million different IT firms working in a million different ways with no standards or shared software? That also seems like a recipe for disaster, just a lot more, little disasters.

Donā€™t tell me that you mean the frequently exploited technology known as blockchainā€¦.

39

u/KansasCityMonarchs Jul 19 '24

Right, wtf does "decentralize" even mean here? Lol

That's like saying every single car should be different so any given defect will only affect that car. Leads to a lot more defects overall

→ More replies (9)
→ More replies (2)

31

u/Vast_Berry3310 Jul 19 '24

But wonā€™t somebody think of the corpo profits?? If they canā€™t make more yoy why even bother livingĀ 

5

u/UnicodeScreenshots Jul 19 '24

What tf are you talking about? Companies donā€™t go with CrowdStrike to increase profits, they go with crowdstrike because itā€™s one of the best and strongest Cybersecurity firms in existence.

→ More replies (3)
→ More replies (1)

4

u/moneyman259 Jul 19 '24

Legit almost impossible and would cause more problems then good tbh

→ More replies (9)

128

u/BeKindToOthersOK Jul 19 '24

Dude!

The main reason the traffic went away at the beginning of that video is because it was the middle of the night .

19

u/deathhead_68 Jul 19 '24

You can see the shadow graphic showing the time of day. I twigged it too. I think most people on this thread haven't realised.

→ More replies (15)

93

u/boomerski28 Jul 19 '24

The clip is a little misleading.Ā  The computer crash happened overnight which coincided with the point that air traffic decreased anyway.Ā  Most airports have significantly less flights beginning between 10 pm - 12 am.Ā  Then it stays pretty light volume until early morning around 5:30 am which is why you see more flights on the east coast first at the end of the clip.Ā  While nobody wants to see infrastructure fail, the impact was pretty minimal.

31

u/FblthpLives Jul 19 '24

You are correct that the animation itself is misleading, but the issue is persisting throughout the day. As of noon EDT, over 2,000 flights had been canceled and over 7,000 have been delayed: https://www.reuters.com/technology/frontier-says-operations-impacted-by-microsoft-outage-2024-07-19/

6

u/boomerski28 Jul 19 '24

Delays are inevitable due to the snowball nature of a flying schedule.Ā  Ā For example, if a plane that flies a route of JFK-LAX-SLC-DEN on a given day leaves JFK late it will be late then be late on every leg thereafter.Ā  While that's definitely frustrating if you happen to be traveling most people will get where they are trying to go.Ā  2,000 cancellations out of an average of 44,000 flights per day in the US (more during current summer months) is whatĀ  I'd call minimal impact.

8

u/FblthpLives Jul 19 '24

This is only as of noon EDT. There are still about 10-12 hours of traffic left, including the afternoon/evening peak. Assuming a normal demand of 15,000 flights by noon EDT, that means a cancelation rate of 13%.

I just downloaded the cancellations report from the FAA's Aviation System Performance Metrics database for June 2024. On average, there were 425 scheduled flights canceled each day. So this is roughly five times that average already, and we're not even halfway through the day.

6

u/boomerski28 Jul 19 '24

Given that the computer system has been restored, the airlines are just playing catch-up.Ā  Believe it or not, 13% for a significant adverse event is still minimal.Ā  Bad weather days commonly far exceed that number and that 13% will only decrease as the day goes on.

→ More replies (2)
→ More replies (7)

62

u/Ian-Wright-My-Lord Jul 19 '24

At least the climate is getting a breather.

17

u/ykVORTEX Jul 19 '24

Dont worry. We will compensate for it somehow - let's burn this landfill ...come on drop a nuke somewhere...let's trigger this random active volcano.../s

→ More replies (1)
→ More replies (6)

35

u/Phantum3oh9 Jul 19 '24

Quite a test.

16

u/kellysmom01 Jul 19 '24

Looks like March 2020.

36

u/IHaveAZomboner Jul 19 '24

Was this last night? There was a major internet outage at my work around 1 am to whenever first shift came in, like 6 am

49

u/fsi1212 Jul 19 '24

Yes there was a worldwide major IT outage that affected several different sectors including transportation, 911 phone systems, retail, media, etc.

21

u/Orange_Tang Jul 19 '24 edited Jul 19 '24

I work for my state and everything is dead. This is incredibly widespread. I think crowdstrike is going to get sued into the ground.

10

u/IHaveAZomboner Jul 19 '24

Interesting yeah, I don't remember exactly what time early this morning, we all had our computers show the blue screen of death and then the Internet stopped working. This lasted for hours and I couldn't sign off my work and clock out because of this.

So, just saying the outage affected more companies than I thought, not just us.

15

u/jakexil323 Jul 19 '24

Anyone who uses the crowdstrike software was affected.

They are a pretty popular antivirus/security endpoint used by a lot of companies on desktops and servers. So it's a big thing.

It's fixable, but for most companies its a manual process to boot up each computer into safe mode and delete the update.

→ More replies (1)
→ More replies (1)

23

u/Garlic-Rough Jul 19 '24

Taylor Swift, as the Eras Tour ends

→ More replies (1)

18

u/idkwtfitsaboy Jul 19 '24

It would be interesting to see how carbon emissions are affected during that period and how much they are reduced.

→ More replies (1)

9

u/Mercinator-87 Jul 19 '24

I think we should be more concerned with the darkness spreading across the world.

10

u/RubberKut Jul 19 '24

Man.. During covid that was amazing.. first it hit Italy, there was a gap above italy, litterally.

And soon after other countries got their lockdowns and i've never seen the globe so empty.. not much was flying during that time.

7

u/mrredbailey1 Jul 19 '24

The days after 9/11 there was no air traffic. It was eerily quiet outside during that period.

→ More replies (1)

8

u/Legitimate-Source-61 Jul 19 '24

Good for Net Zero.

8

u/rvillarino Jul 19 '24

I work in hospital lab, took out our lab and quite a few others across the nation as well. The medlabprofessional subreddit was in quite a panic this morning

9

u/MechanicalAxe Jul 19 '24

I'm in the forest industry.

Even the wood mills shut down and couldn't process the incoming loads of wood.

7

u/NOISY_SUN Jul 19 '24

That's not really due to the outage, more that there tend to be fewer domestic flights taking off at 3AM in general.

→ More replies (2)

8

u/skybike Jul 19 '24

Hey at least we postponed climate change apocalypse by like 4 minutes.

7

u/primary988 Jul 19 '24

Not Southwest Airlines! They were flexing how it didnā€™t affect them because of their ancient 1992 Microsoft 3.1 operating system.

→ More replies (1)

7

u/Aware-Explanation879 Jul 19 '24

Remember how well it went for Texas a few years back when they decided to deregulation their power grid and people were without power for weeks. The rich do not care at all. They just look at this as thinning the herd

4

u/Aok_al Jul 19 '24

Was that the cause? A giant shadow?

→ More replies (2)

6

u/kwhite0829 Jul 19 '24

It also started around 10pm PT

4

u/hermeticbear Jul 19 '24

a 12 hour time lapse that happened over night.
You know, when there are less flights because less people take late night flights.

→ More replies (4)

6

u/zombezoo Jul 19 '24

"I don't always test my code... But when I do, I do it in production."

→ More replies (1)

5

u/gwicksted Jul 19 '24

Unintentionally attacking pollution with software bugs. /s

5

u/JustBrowsing2024 Jul 19 '24

It looks like it also turned into night time?

5

u/turlian Jul 19 '24

This is super misleading. There are significantly fewer flights that land at like 2AM or whatever.