CrowdStrike Falcon Causing BSOD? Here’s How to Fix It

Updated 7/19/2024 8:55AM MST

CrowdStrike, a leading cybersecurity company, confirmed that an issue with its Falcon Sensor software is causing Windows systems to crash. The Falcon Sensor is an endpoint detection and response system designed to prevent computer systems from cyber attacks. However, a recent update to this software has resulted in widespread crashes, with systems displaying the Blue Screen of Death.

"CrowdStrike is aware of reports of crashes on Windows hosts related to the Falcon Sensor," the company wrote in an alert at 1:30 a.m. ET on Friday. "Symptoms include experiencing a bugcheck\blue screen error related to the Falcon Sensor. Our Engineering teams are actively working to resolve this issue and there is no need to open a support ticket."

Global Impact

The problem is not isolated, with reports of outages coming in from several countries, including Australia, New Zealand, India, Japan, and the UK. In the U.S., crowdsourced outage monitor DownDetector indicated a spike in issues across various platforms and businesses, including Amazon Web Services, Instagram, eBay, Visa, ADT, and PlentyOfFish. Several U.S. airlines such as American, Delta, and United Airlines were grounded due to communication issues seemingly related to the CrowdStrike update.

Supermarkets, banks, airlines, telecommunications companies, and TV broadcasters are among the businesses impacted. For example, shoppers in Australia were unable to pay for groceries due to electronic payment systems being down. Sky News in the UK was temporarily off-air, and the London Stock Exchange experienced technical issues with its RNS news service.

CrowdStrike Statement on Falcon Content Update

CrowdStrike has issued a detailed statement addressing the defect found in a recent content update for Windows hosts, emphasizing that this is not a security incident or cyberattack. The issue has been identified, isolated, and a fix has been deployed. Mac and Linux hosts remain unaffected.

CrowdStrike is actively working with impacted customers and recommends consulting the support portal for the latest updates. The company underscores the importance of communicating through official channels to ensure accurate information.

Update 9:22am ET, July 19, 2024:

CrowdStrike's engineering team has identified the problematic content deployment and reverted the changes. They are committed to providing continuous updates and have published a CrowdStrike Tech Alert with detailed information and workaround steps.

CrowdStrike has identified and addressed the issue causing Windows crashes and provided workaround steps for affected systems. For individual hosts, users should reboot the system to allow the reverted channel file to download. If the system crashes again, they should boot into Safe Mode or the Windows Recovery Environment, navigate to the CrowdStrike directory, and delete the problematic file "C-00000291*.sys" before booting normally. For BitLocker-encrypted hosts, a recovery key may be required. In cloud or virtual environments, users can either detach the operating system disk, delete the problematic file by attaching the disk to another server, or roll back to a previous snapshot before the issue occurred. These steps ensure that systems can return to normal operation while CrowdStrike continues to work on a permanent fix.

Response and Mitigation Efforts

At 2:30 a.m. ET, CrowdStrike announced that it had identified the update causing the issue and rolled it back. The company also offered a workaround for those still experiencing problems:

How to Fix CrowdStrike BSOD

To address the Windows Blue Screen of Death (BSOD) issue caused by the recent CrowdStrike update, you can follow a temporary workaround provided by CrowdStrike. This involves booting Windows into Safe Mode or the Windows Recovery Environment, navigating to the System32\drivers\CrowdStrike directory, locating the file matching "C-0000029*.sys," and deleting it. After this, you can boot your system normally. However, this is only a temporary solution, and CrowdStrike is actively working on a permanent fix to resolve the root cause of the issue. It is important to note that it is currently unknown if this workaround disables CrowdStrike Falcon's protective features, so users should stay updated with CrowdStrike's communications for further instructions and a complete resolution.

Workaround Steps For Individual Hosts

1. Boot into the Windows Recovery Environment (WinRE).
2. In the WinRE menu, select "Troubleshoot," then "Advanced options," and finally "Command Prompt."
3. In the Command Prompt, type the following command to navigate to the CrowdStrike directory:

cd C:\Windows\System32\drivers\CrowdStrike

5. In the Command Prompt, type the following command to list the file in the CrowdStrike directory:

dir C-0000029*.sys

6. To delete the problematic file, type the following command.

del C-0000029*.sys

7. Once the file is deleted, close the Command Prompt and restart your computer.

Workaround Steps for Public Cloud

Option 1

• ​​​​​​​Detach the operating system disk volume from the impacted virtual server
• Create a snapshot or backup of the disk volume before proceeding further as a precaution against unintended changes
• Attach/mount the volume to to a new virtual server
• Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory
• Locate the file matching “C-00000291*.sys”, and delete it.
• Detach the volume from the new virtual server
• Reattach the fixed volume to the impacted virtual server

Option 2

• Roll back to a snapshot before 0409 UTC.

While this workaround is effective, it requires manual intervention on each affected system, which can be time-consuming for large organizations with many computers.

Retrieving Your BitLocker Recovery Keys

When performing the steps to fix the CrowdStrike-induced Windows BSOD issue, you may be prompted for BitLocker recovery keys. If you don't have these keys readily available, you can retrieve them from your Microsoft account. Here’s how:

1. Access Your Microsoft Account:
   • Open a web browser and go to Microsoft's Device Recovery page.
2. Sign In:
   • Sign in with the Microsoft account credentials associated with your device.
3. Locate Your Device:
   • Once signed in, you will see a list of devices associated with your account.
   • Find the device that is asking for the BitLocker recovery key.
4. Retrieve the Recovery Key:
   • Click on the link to view the recovery key for your device.
   • Note down the 48-digit recovery key displayed.
5. Enter the Recovery Key:
   • When prompted on your device, enter the 48-digit recovery key to unlock your BitLocker-encrypted drive.

By following these steps, you can quickly retrieve your BitLocker recovery keys and proceed with the necessary actions to resolve the CrowdStrike issue.

Broader Implications

The timing of this incident, occurring early Friday morning in North American time zones, adds significant stress to IT teams, particularly as they head into the weekend when availability is typically lower. This situation is likely to exacerbate the challenges faced by IT departments already stretched thin, as they now need to address urgent system recoveries with limited resources. For remote employees, IT support will be required to guide them through the recovery steps over the phone, further taxing corporate IT resources at a time when they are least prepared to handle such a widespread issue.

Additionally, this outage is expected to have a measurable impact on companies relying on cloud infrastructure running CrowdStrike Falcon. Accessing recovery screens is not always straightforward or even possible in certain scenarios, complicating the remediation process.

The strain on IT teams and the complexity of the required workaround highlight the critical need for a permanent fix from CrowdStrike to prevent similar disruptions in the future.

Conclusion

The CrowdStrike outage serves as a reminder of the importance of robust disaster recovery plans and the need for companies to be prepared for unexpected technical issues. As businesses around the world work to mitigate the impact of this outage, the incident underscores the critical role of cybersecurity in maintaining operational stability.

Organizations affected by the outage should follow CrowdStrike's recommended workaround to restore functionality and monitor communications from the company for further updates. In the meantime, businesses should review their IT infrastructure and consider additional safeguards to prevent similar issues in the future.