CrowdStrike and Microsoft Incident: The biggest breach ever. Let’s take a dig now:
CrowdStrike
CrowdStrike is a leading cybersecurity firm known for its innovative approaches to endpoint protection, threat intelligence, and incident response. Founded in 2011, CrowdStrike has quickly become a prominent name in the cybersecurity industry, offering cutting-edge solutions to combat advanced threats. The company’s flagship product, the Falcon platform, is highly esteemed for its robust threat detection and response capabilities.
Falcon utilizes cloud-native technology and artificial intelligence to provide comprehensive protection against a wide range of cyber threats. Its advanced detection mechanisms enable real-time monitoring and automated threat responses, significantly reducing the time it takes to identify and mitigate potential security breaches. CrowdStrike’s emphasis on proactive threat hunting and in-depth threat intelligence allows organizations to stay ahead of emerging threats.
In addition to its technological prowess, CrowdStrike is known for its expert incident response services. The company’s team of seasoned cybersecurity professionals offers rapid and effective responses to security incidents, helping organizations minimize damage and recover swiftly. Through a combination of innovative technology and expert services, CrowdStrike continues to set high standards in the field of cybersecurity.
Microsoft
Microsoft, founded in 1975, is a global technology powerhouse with a vast array of products and services spanning multiple industries. Renowned for its software solutions, such as Windows and Office, Microsoft has also made significant strides in cloud computing and cybersecurity. Microsoft’s Azure cloud platform is a dominant force in the cloud computing market, offering a comprehensive suite of services that cater to businesses of all sizes.
Azure provides a scalable and secure cloud infrastructure, enabling organizations to build, deploy, and manage applications efficiently. With a focus on security, Azure incorporates numerous features designed to protect data and applications from a variety of threats. Microsoft’s cybersecurity solutions extend beyond Azure, encompassing a wide range of products and services aimed at enhancing security across its ecosystem.
In addition to its technological offerings, Microsoft is committed to advancing cybersecurity through collaboration and innovation. The company invests heavily in research and development, continuously improving its security technologies to address evolving threats. Microsoft’s proactive approach includes working closely with industry partners, governments, and other stakeholders to enhance global cybersecurity standards.
CrowdStrike and Microsoft Incident: Overview of the Biggest IT Outage ever recorded
Several million Windows systems around the world were affected by the largest IT outage in history i.e. CrowdStrike and Microsoft Incident, caused by a botched CrowdStrike software update. According to insurers, the outage will cost approximately $5.4 billion to Fortune 500 companies.
It is CrowdStrike, an endpoint security vendor whose Falcon platform is at the core of the outage. To minimize cybersecurity risks, this helps protect systems against potential threats.
Firstly, we want to inform you that this is not a cyber attack or a cyber breach. A software update released by CrowdStrike, a renowned cybersecurity company, affected IT systems worldwide on July 18. Crowdstrike’s update causes Blue Screen of Death on about 8 million Windows machines.
What is the “Blue Screen of Death”?
When Windows shuts down or restarts unexpectedly due to a serious problem, blue screen errors (also called black screen errors) occur. Messages such as “Windows has been shut down to prevent damage to your computer” may appear.
What caused the epic outage of CrowdStrike and Microsoft Incident?
The CrowdStrike Falcon platform is used by organizations of all sizes and across a wide range of industries. CrowdStrike’s technology’s pervasiveness and integration into so many mission-critical operations and industries amplified the effect.
It was not a flaw in Microsoft Windows that caused the outage, but rather a flaw in CrowdStrike Falcon.
Falcon interacts with the Microsoft Windows kernel as a Windows process. It can monitor operations across the OS in real time thanks to its high privileges. Falcon sensor version 7.11 and above crashed due to a logic flaw. CrowdStrike Falcon’s tight integration into the Microsoft Windows kernel caused a BSOD and a Windows system crash.
A sensor configuration update contained the flaw in CrowdStrike Falcon. In order to protect users from threats and mitigate threats, the sensor is regularly updated — sometimes multiple times per day.
“Channel files,” which provide configuration updates for behavioral protections, contained the flawed update. The channel file 291 was supposed to help improve Falcon’s evaluation of named pipe execution on Microsoft Windows. On Microsoft Windows, named pipes are a common method of interprocess communication.
CrowdStrike introduced a logic error in channel file 291, causing the Falcon sensor and Windows systems that used it to crash.
This flaw isn’t present in all versions of channel file 291. The problematic version is channel file 291 (C-00000291*.sys) with UTC timestamp of 2024-07-19 0409. The logic flaw is not present in channel file 291 timestamped 2024-07-19 0527 UTC or later. By then, CrowdStrike had reverted the change after discovering its error. For many of its users, that reversion came too late as they had already updated, resulting in BSODs and inoperable systems.
In spite of the fact that this was not a Microsoft incident, Microsoft provided updates on the steps they have taken with CrowdStrike and others to remediate and support their customers.
CrowdStrike and Microsoft Incident: Throughout this event, Microsoft has kept in touch with its customers, CrowdStrike, and external developers to collect information and expedite solutions. Several businesses and individuals have been disrupted by this problem. Technical guidance and support are provided to customers to bring disrupted systems back online safely. A number of steps have been taken, including:
- Engaging CrowdStrike to automate their solution development process. The company has also issued a public statement recommending a workaround. The Windows Message Center was updated with instructions for fixing the problem on Windows endpoints.
- Assisting customers directly with the restoration of services through the deployment of hundreds of Microsoft engineers and experts.
- Collaboration with other cloud providers and stakeholders, including Google Cloud Platform (GCP) and Amazon Web Services (AWS), to share industry awareness and inform ongoing conversations with CrowdStrike and customers.
- Documentation and scripts found on their website about manual remediation should be posted as soon as possible
- By using the Azure Status Dashboard, customers are kept informed of the latest incident status
In what ways were services affected?
A logic error flaw in CrowdStrike affected approximately 8.5 million Windows devices. Microsoft has less than 1% of the world’s Windows installations.
While the affected systems were a small proportion of the overall Windows install base, they ran critical operations. The following services are affected.
1. Airlines and airports
More than 10,000 flights around the world were delayed or canceled as a result of the outage. Delta, United, and American Airlines were among the affected airlines in the United States. As a result, hundreds of flights were canceled by these airlines until their systems were restored. Multiple airlines and airports were affected globally, including KLM, Porter Airlines, Toronto Pearson International Airport, Zurich Airport, and Amsterdam Schiphol Airport.
2. Public transit
In addition to Chicago, Cincinnati, Minneapolis, New York City, and Washington, DC, public transportation was disrupted in multiple cities.
3. Healthcare
Globally, appointment systems were disrupted, causing delays and cancellations in hospitals and clinics. Alaska, Indiana, and New Hampshire reported that 911 emergency services were also affected.
4. Financial services
Financial institutions and online banking systems around the world were affected by the outage. Several payment platforms were affected, and some individuals did not receive their paychecks on time.
5. Media and broadcasting
The outage affected a number of media and broadcast outlets around the world, including Sky News in the United Kingdom.
Our broad ecosystem – global cloud providers, software platforms, security vendors and other software vendors, and customers – is interconnected. As a result, it is a reminder of how important it is for all of us across the tech ecosystem to prioritize safe deployment and disaster recovery. Over the last couple weeks, we’ve seen that we learn, recover, and move forward most effectively when we collaborate with each other.
Is there a reason why Apple and Linux were not affected?
Besides Microsoft Windows, CrowdStrike’s software also runs on macOS and Linux.
However, only Microsoft Windows was affected by the July outage. Windows systems were affected by a faulty sensor configuration update that caused the outage. As named pipe execution only occurs on Microsoft Windows, the channel file 291 update was never released to macOS or Linux systems.
It is also different in macOS and Linux how the Falcon sensor integrates as a kernel process. To limit potential risks, those OSes have different integration points.
It was reported in June from Linux vendor Red Hat that the Falcon sensor running as an eBPF program in Linux triggered a kernel panic. A kernel panic is similar to a BSOD in Linux, but less dramatic. Red Hat reported no major incidents related to that issue.
After this outage, how long will it take businesses to recover?
It took CrowdStrike 79 minutes to identify and deploy a fix for the issue. Despite CrowdStrike’s rapid identification and deployment of a fix, businesses’ recovery processes are complex and time-consuming. The problem was that, once the problematic update was installed, the underlying Windows OS would trigger BSODs, rendering the system inoperative.
To restore normal operations, IT administrators had to manually boot affected systems into Safe Mode or Windows Recovery Environment. The process is labor-intensive, especially for organizations with many affected devices. Physical access to each machine was also required in some cases, adding further time and effort.
In some cases, the fix could be applied within a few days. For those with extensive IT infrastructure and encrypted drives, the process was not straightforward. Because some organizations used the Microsoft Windows BitLocker encryption technology, recovery was significantly more time-consuming.
Some organizations could take months to fully restore all affected systems after the outage.
In the event of a technical outage, how can businesses prepare better?
Due to today’s heavy reliance on technology, the CrowdStrike Windows outage highlighted the vulnerabilities of modern society. Manual procedures can significantly enhance business continuity during tech outages, even if backups and automated processes are in place.We suggest to our patronage that they include this kind of outage scenario moving on.Enhance the importance of DRBCP in their business and how effectively can they manage the impact and go back to business as usual.
Is this article of interest to you? How far has your organization progressed in terms of Disaster Recovery and Business Continuity? You have reached the end of your search. Get in touch with us today to learn more about our Disaster Recovery and Business Continuity services! Please feel free to contact us at info@res-q-rity.com.
Click here, to visit us at Res-Q-Rity and book an appointment with us.
