A global outage taking down a wide range of Microsoft 365 services remains ongoing Friday morning.
The outage started midday Thursday, impacting users of Microsoft email, cloud, and AI services. Microsoft has documented the full list of services impacted by the error, while moving early Friday morning to protect customers around the world from losing emails during the extended disruptions.
“Recovery actions are ongoing,” the Microsoft 365 Status account confirmed on X. “To help preserve delayed mail during this event, we’ve extended email timeout to 48 hours.”
Similar to recent world-shaking outages of widely used tech like Amazon Web Services and CrowdStrike, the Microsoft disruption looks like “one of the biggest outages of all time,” Ciaran Martin, the former head of the UK’s National Cyber Security Centre, told NBC News. Possibly even “worse,” he suggested.
Businesses, hospitals, police departments, and schools were among those scrambling to work around the outage. In Japan, it disrupted government IT infrastructure, according to deputy chief cabinet secretary Masanao Ozaki. NBC’s report documented various harms, noting that “few public bodies were immune from the outage.” In some cases, doctors couldn’t send patients “important” updates on medication changes or follow-up care.
The financial sector suffered, too, with people blocked from making bank transfers since they couldn’t receive verification codes.
Justice systems were also hobbled. The Supreme Court in the Philippines saw its legal work derailed, NBC reported. And one US sheriff’s office in Colorado reported that the outage made it difficult for people to post bond for prisoners’ immediate release.
It’s unclear how many users were affected, but Microsoft estimated in January that the total number of paid Microsoft 365 commercial subscriptions exceeded 450 million.
Microsoft blames failed server health checks
On a status page, Microsoft said “we have no evidence of malicious activity associated with this incident” and promised to “share a full Post-Incident Report with root cause and defense-in-depth actions after mitigation is complete.”
“We’re aggressively pursuing all pathways to reduce impact,” Microsoft said on X.
But Microsoft’s public statements make clear that the tech giant was blindsided by the outage and struggled for hours to find mitigations that could reliably restore user access.
Shortly after detecting the outage, Microsoft confirmed that the main issue was “a portion of infrastructure” that “is experiencing elevated load across multiple regions.” This was not caused by increased traffic, Microsoft said, but “a problem that’s causing servers to fail automated health checks and become unavailable.”
Once it became clear that the failed health checks were to blame, Microsoft said it took steps to prevent additional server outages while validating alternate health check methods. But four hours later, the company admitted that it was still in the weeds.
“We’re opening all available pathways to restore service, as our actions taken thus far have not provided the anticipated relief,” Microsoft said. “We found a core process on affected servers is becoming overwhelmed, causing capacity loss. To validate a potential recovery, we’re isolating a portion of infrastructure, restarting affected components, and reintroducing traffic to confirm the services remain stable before applying the change more broadly.”
Even small wins seemed to turn into new setbacks. As Microsoft worked to resolve the issue, some of the servers it brought back online would “become unresponsive again once they receive traffic,” giving the company yet another problem to solve.
Realizing that no quick solutions were coming, Microsoft moved to extend email timeout to 48 hours. As “encouraging” recovery efforts remained ongoing, Microsoft advised users that “some intermittent issues may continue” when traffic is restored.
Users upset by Copilot, Outlook issues
Microsoft’s most recent updates confirmed that it’s seeing “broader improvements in North America, sustained improvements across much of Europe, and encouraging recovery signals in Asia Pacific.”
“Some users may still experience intermittent access issues or delayed email delivery as mitigation continues across remaining affected infrastructure,” Microsoft said. “Our focus remains on stabilizing recovered infrastructure, completing mitigation in remaining impacted areas, and reducing delayed email queues and backlogs while we validate service consistency across connection methods.”
DownDetector, which tracks user reports of service issues globally, showed that connectivity losses peaked Thursday afternoon and dropped off substantially by Friday morning.
In comments, an Outlook user described the pains of monitoring email recovery, complaining that outgoing messages seem to send fine but incoming messages are received inconsistently. Some Copilot users seemed distressed that access wasn’t restored as quickly as Outlook. One Microsoft 365 user pondered, “How many offices had people with practically nothing to do? How much has this cost?”
Ashley BelangerSenior Policy Reporter
Ashley BelangerSenior Policy Reporter
Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.
Source: arstechnica.com
