CrowdStrike's "Blue Screen of Death" Disaster: A Billion-Dollar Revelation on the Importance of Backup and Recovery Systems

This incident has not only highlighted the importance of backup systems to the industry but also demonstrated that software solutions become unusable when the operating system fails. This suggests that, in future remote management solutions for devices, users might increasingly rely on subsystems for repair and system restoration, in order to enhance cyber resilience thoroughly.

Thousands of engineers had to be deployed on-site for system restoration

Crowdstrike, the world’s second-largest endpoint security software provider and a NASDAQ-listed company, is trusted by 62 of the Fortune 100 companies as their cloud security provider and serves nearly 30,000 global enterprise customers. The system that caused the disaster was their flagship product, Falcon.

Antivirus software requires integration at the core level of the operating system to provide comprehensive protection. This is why a failed update caused the entire operating system to enter an endless reboot loop. As a result, over 8 million devices worldwide were paralyzed for several hours, disrupting operations in various industries, including aviation, finance, retail, healthcare, and transportation.

Approximately 250,000 devices remain out of service to date. Since the devices could not enter the operating system, Microsoft had to deploy 5,000 engineers to the field to assist with restoring device operations, with some devices requiring dozens of restart attempts before resuming operation. Although Microsoft estimates that the affected devices account for only about 1% of its global devices, the initial projection of losses across various industries reaches billions of US dollars.

OOB Remote Management could accelerate disaster recovery

With the growing popularity of digital transformation and edge AI in recent years, businesses across all industries are compelled to adopt digital technologies to enhance their operations, a process that must be approached with caution. Therefore, this incident drives organizations to more actively explore viable backup solutions in case of similar future events. These solutions include hybrid cloud deployments and the use of backup servers from different cybersecurity vendors.

Allxon has long focused on the remote management of edge AI devices. The Out-of-Band (OOB) software-hardware-cloud remote management solution, integrated with Nuvoton’s microcontroller (MCU), NUC980, allows for remote disaster recovery for a large number of devices by installing a small OOB hardware module in the device as an independent backup system. If faced with a situation like the CrowdStrike incident, users with Allxon OOB solution could employ the following methods to restore their devices:

1. Allxon Cloud Serial Console: If the device is equipped with a backup boot device and has integrated Allxon OOB with the Cloud Serial Console, users  can remotely access the BIOS to modify the boot drive settings. After successfully entering the operating system, users can repair the faulty system.

2. SSD Recovery & Backup: The SSD Recovery & Backup feature, co-developed by Allxon and Apacer, allows remote system recovery via OOB using data backed up in another partition of the main system SSD.

Learn More About Allxon's solution

With more devices moving to the edge in the future, Allxon is ready with software-hardware-cloud service capabilities

In fact, recognizing the crucial role of digital signage in delivering real-time information in places like airports, some customers have already implemented Allxon OOB for digital signage in airport stores across Europe. Today, the edge AI market continues to grow and is expected to expand from its current scale of tens of billions to hundreds of billions of USD over the next decade¹. This means more devices will be deployed at the edge, leading to potentially greater losses caused by device failures.

We at Allxon, with our technical capabilities in software, hardware, and cloud services, has established deep partnerships with software and hardware suppliers in the edge AI application sector. Allxon will continue to play a connector role in the industry, designing more secure, stable, manageable, and operationally efficient edge AI solutions for system integrators (SI) and managed service providers (MSP) across various application fields.

Reference