Microsoft Outlook outage: a July code change caused it
Microsoft says the Exchange Online and Outlook outage came from a defect it shipped in July. It sat quietly for months, and a rollback was not an option.

The Microsoft Outlook outage that left Microsoft 365 users without email last week traces back to a code change from July. For months it did nothing. Then, on November 12, it took down most of a network gateway fleet.
According to a preliminary post incident report Microsoft published on November 15, the update had passed load testing and rolled out without any visible impact. The problem: "Our load testing tended to focus on weaker hardware with fewer cores." The defect caused lock contention (processes fighting over the same resource) inside a Windows process, which used up CPU and memory. Microsoft says it gets worse on machines with more cores and newer hardware.
What triggered the Exchange Online outage?
Two ordinary things happened together. Teams failed over to the primary gateway that supports Exchange Online, which created a load surge. At the same time, almost 14 percent of the gateway's server capacity was out of service and on its way back. Microsoft says there was enough headroom for either event alone. Together they hit the defect.
Then came a cascade. When a server stopped responding, traffic moved to the next healthy one, which hit the same bug. Within two hours the defect had spread across the fleet, and at the lowest point more than 80 percent of servers were unresponsive.
Why was it so hard to fix?
Restarting servers did not help, because they were overwhelmed again straight away. Microsoft's automated recovery "was not designed to restore service from this broad unresponsive state," and a simple rollback was off the table because the defect was months old. Engineers wrote a patch for the patch: an update returning the code to its previously known good state in Windows, applied to groups of servers one by one.
Teams, OneDrive, SharePoint and Office apps could fail out of the broken gateways. Exchange Online could not, for scale reasons, which is why email stayed down longest.
The timeline: the Teams failover came at 0107 UTC on November 12, Microsoft's monitor flagged connectivity problems at 1603 UTC, and the first support case arrived at 1633 UTC. The fix was applied at 0720 UTC on November 13. Microsoft confirmed healthy service at 1610 UTC and sent its final customer update at 0415 UTC on November 14. A final report is due on November 20.
Apps worth a look
Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.
Talk to an AI conversation partner; every sentence comes back corrected and explained in your language. 9 languages, A1 to C2.
Open EDIFACT and X12 messages, see every segment explained, convert to JSON/XML, export PDF and Excel. Runs locally, no upload.
Pick who it is for, the occasion and their personality. The search engine reads product descriptions from local shops and ranks what fits. Free, no sign-up.
More from this day
See all →
Comet 220P/McNaught suddenly got 1,000 times brighter. Here's why

Coral Hart used AI to write 370 books in 20 months

WhatsApp Zaps: self-deleting photos and videos for any chat

Green SM taxis: Karnataka halts new registrations, eyes mandatory meters
More in: Web & Dev
See all →
India unblocks 95 websites after fake Allahabad High Court order

AO3 is back online after two weeks, with some features missing

LinkedIn Frequent Viewer: a test shows who keeps visiting you
