A massive Microsoft Windows BSOD (blue screen of death) outage has impacted multiple companies worldwide including airlines, broadcasters and others. The problem was caused by a faulty update from security giant CrowdStrike that forced PCs and servers into an unrecoverable boot loop. The change has since been rolled back, and airlines and hospitals were gradually recovering by midday in the US, but many machines are still affected.
“We have widespread reports of BSODs on windows hosts, occurring on multiple sensor versions,” CrowdStrike wrote in a pinned Reddit post. “[We have] identified a content deployment related to this issue and reverted those changes.” The company went on to describe a workaround, which involves booting Windows into Safe Mode and deleting a specific driver. CrowdStrike CEO George Kurtz apologized for the global meltdown on the Today show (via The NY Times), saying, “We’re deeply sorry for the impact that we’ve caused.”
The issue forced Delta, Frontier and other airlines to ground flights, and impacted UK broadcaster Sky and the London Stock Exchange. On a Reddit thread, dozens of commenters stated that their companies were effectively offline due to the problem. Flight-tracking service FlightAware reported in mid-afternoon that over 2,500 US flights had been canceled on Friday. They gradually recovered as the day continued, but the restoration was far from complete.
12-hour timelapse of American Airlines, Delta, and United plane traffic after what was likely the biggest IT outage in history forced a nationwide ground stop of the three airlines. pic.twitter.com/wwcQeiEtVe
— Colin McCarthy (@US_Stormwatch) July 19, 2024
US Transportation Secretary Pete Buttigieg told airlines on Friday that they would need to handle the situation as if it were a self-inflicted (mechanical or technical) failure, which requires them to cover travelers’ food, transportation and lodging costs for those whose delays last longer than three hours. Earlier in the day, United Airlines and Delta had told stranded airline passengers they’d have to foot the bills themselves since the CrowdStrike meltdown was out of their control. A United spokesperson later reversed its previous stance after Buttigieg’s comments.
IT pros around the world struggled to adapt to the nearly impossible hand they’d been dealt.
“Even if [CrowdStrike] fixed the issue causing the BSOD, I’m thinking how are we going to restore the thousands of devices that are not booting up,” one user noted. “Let me explain to someone who is not tech savvy and is working from home how to boot their machine into safe mode,” another wrote.
Reddit users including many IT managers from Australia, Malaysia, Japan, India, the Czech Republic and elsewhere have said they’re fighting through the issue. “Here in the Philippines, specifically in my employer, it is like Thanos snapped his fingers. Half of the entire organization [is] down due to BSOD loop. Started at 2pm and is still ongoing. What a Friday,” said one.
US court systems were affected as well, with cases scheduled for the day postponed (including the latest trial of the disgraced Hollywood mogul Harvey Weinstein). Many hospitals reportedly postponed surgeries requiring anesthesia, and some were unable to reschedule due to the required appointment planning software also being down.
CrowdStrike is a US-based security firm that provides real-time protection against security threats to corporations. One of its key products is Falcon, described by the company as “providing real-time indicators of attack, hyper-accurate detection and automated protection” from threats. A CrowdStrike spokesperson said it was likely an issue with Falcon that caused the incident.
To add to the pain, Microsoft appears to have also suffered a separate outage with its Azure services and Microsoft 365 app suite. “Users may be unable to access various Microsoft 365 apps and services,” it wrote. It’s not clear which, if any, of the outages are related to this instead of the CrowdStrike problem.
Update July 19, 2024 6:12 AM ET: CrowdStrike CEO George Kurtz has acknowledged the problem on X, saying it was caused by a defect in a content update for Windows hosts, and not a “security incident or cyberattack,” He added that “the issue has been identified, isolated and a fix has been deployed.” There’s no mention on whether the fix will be usable on machines currently stuck in a boot loop.
Mac and Linux hosts are not impacted, he added. An identical statement has been posted on CrowdStrike’s blog.
Update, July 19, 2024, 3:46 PM ET: This story has been updated to add developments around flights, hospitals, court cases and an apology from CrowdStrike’s CEO.
Credit: Source link
