Osegi
Software·2 min read

Microsoft 365 outage: a bug from July and a routine failover

Microsoft says the November 12 Microsoft 365 outage began with a bug that sat unnoticed for months, then met a routine failover and missing server capacity.

Rows of servers in a dark data center with a red warning light glowing on one rack.
Image: mint

The Microsoft 365 outage on November 12 was not caused by one big mistake. It took three small things landing at the same time, and one of them had been sitting in the code since July.

According to Charles Lamanna, Microsoft's Executive Vice President of Copilot, Agents and Platform, the combination overwhelmed the company's networking gateways. At the worst point, more than 80% of the affected gateway servers stopped responding. Teams, Exchange Online, OneDrive, SharePoint and Office apps were all hit.

What caused the Microsoft 365 outage?

First, a latent bug. A code change made for Microsoft 365 networking gateway servers contained a lock contention problem, meaning processes competing for the same resource until the server chokes. It was deployed in July 2026 and rolled out worldwide over several weeks. Nothing visibly went wrong for more than three months. Microsoft says its load tests used different traffic patterns and mostly older, weaker hardware, while the bug hit hardest on newer machines with more processor cores.

Second, a failover. About 15 hours before the outage, Teams switched from Microsoft's alternative gateway to the primary one, which also serves Exchange Online. That is normal behaviour, but it changed the traffic on the primary gateway and made the bug more likely to fire.

Third, nearly 14% of the gateway's server capacity was already offline because of an unrelated problem the day before. That alone was survivable. Together with the other two, it was not.

Why did it spread so far, and take so long to fix?

Microsoft's traffic systems normally send requests from dead servers to healthy ones. Here that made things worse: the healthy servers took on extra load and hit the same bug. In under two hours the failure had cascaded across the gateway infrastructure. Barriers between capacity partitions, which should contain such problems, did not hold because of a separate issue.

Teams, OneDrive, SharePoint and Office could fail over away from the broken gateway, so they came back sooner. Exchange Online could not. Recovery was also awkward: servers that returned were flooded by queued requests and fell over again, and because the change was months old, the usual rollback tools and feature flags could not undo it quickly.

Engineers built an update that restored the earlier code, then took groups of servers offline, patched and restarted them, sometimes more than 50% of a region at once. Microsoft says a final Post-Incident Review with more detail is due later in the week.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:microsoft 365outagemicrosoftteamsexchange onlinecloud

More from this day

See all →

More in: Software

See all →