Todays Global Tech Meltdown: Crowdstrike Update Gone Wrong!

These things seem to be getting all too common, some globally installed software breaks and takes down half the world!

It was only last year that facebook went down and social media interaction as we knew it came to a halt, small businesses couldn't promote their wares, parents couldn't embarrass their children by tagging them in pictures, influencers couldn't upload their influential aspirational photos and people had to actually talk to each other again, for a little while at least.

Even worse, back in 2016 a computer programmer deleted 11 lines of code from an open source repository which started a chain effect taking down masses of computers and software that had included that code in its systems and in 2020 the google network went down for 6 hours changing life as we know it! Well, for 6 hours anyway!

So what's happened today? A mini apocalypse happening right now. Airports, hospitals, you name it – everything's grinding to a halt thanks to a massive software outage. Businesses that rely on Windows are finding their computer crashing with that horrid blue screen of death (you know the one you used to see all the time but rarely do nowadays, well it's back with a vengeance)

You may have problems getting hold of commercial services, getting your flight, paying for your dinner on a debit card and even getting the money out of the cash machine...

So what's going on:

• The culprit? A piece of software from Crowdstrike, they are a massive name in security protection for Windows machines. You may never have heard of them, but it seems that most of the financial and business world use their software, a kind of anti-virus app has suddenly started playing destruction derby with a ton of computer systems. Or in actual terms, Crowdstrike made a software update and rolled it out to all the computers at the same time!
  
  
• Why is this such a big deal? Crowdstrike is HUGE. one-in-five commercial Windows machines globally use their stuff. So... This update went out with a bang...the bad kind.
  
  
• The "fix" is a nightmare. Usually you'd just uninstall or roll back, but these computers are currently like bricks and to fix it Crowdstrike wants all its users to manually reboot every single affected machine into safe mode, delete a file, and pray to the tech gods. Not exactly ideal during a global crisis.

How did this happen? Well here's the real head-scratcher, even when making simple changes to a piece of software, there should be a whole raft of testing procedures in place before "deploying to live". Where was the testing? This wasn't an edge case, it appears to affect every machine that it updated to, that surely should have been caught in the testing? The company has a value of $80B and a 22% market share, they have the money and resource to check the software they create?!?!

Why wasn't this a staged rollout?

Often, software updates are rolled out in phases, you know, test the waters before diving in headfirst. But Crowdstrike went full YOLO (you only live once) and unleashed this update worldwide. At the same time, pulling the kill switch.

This is a major wake-up call for the tech industry. Staged rollouts are NOT optional, especially when your software is mission-critical. We're talking about banks, airports and hospitals for goodness sake!!

The fallout from this mess will be huge, both financially and for Crowdstrike's reputation. Who wants a security company that takes down the very systems it's supposed to protect?

So, let this be a lesson: If you create software of any type then ALWAYS test, test and triple test your updates. And roll them out in a sensible manner. The potential consequences are just too flipping high.