Osegi
Software·2 min read

Microsoft Outlook outage: Nadella blames a bug hidden since July

Satya Nadella says the 24-hour Outlook outage came from a latent bug that only showed up under one traffic pattern, and admits Microsoft should have tested better.

Satya Nadella speaking at a Microsoft event, with an Outlook email inbox on a laptop screen in the foreground.
Image: The Indian Express

The Microsoft Outlook outage that knocked out email for more than a day came down to a bug nobody had spotted for months. Satya Nadella called it "a crazy perfect storm" in an interview on CNBC's Mad Money with Jim Cramer.

According to Nadella, the bug was latent, meaning it sat dormant until Outlook saw a particular pattern of traffic. Under normal conditions everything worked. Under that unusual load, it didn't. "Of course, we should have done the testing before," he said. "We should have figured out how to have the bulkheads such that you didn't have a global outage."

What caused the Microsoft Outlook outage?

Microsoft's preliminary investigation points to three factors: a software bug introduced in July, extra traffic after a Teams failover, and reduced server capacity. Nearly 14 per cent of the affected gateway's capacity was already unavailable. (A failover is when a system automatically switches to a backup after the main one stops working.)

The problem started on November 12 at about 7:40 am PT, with users unable to get into Outlook. It then spread to Teams, Copilot, Microsoft 365, SharePoint and OneDrive. The trouble was traced to Exchange Online, the infrastructure behind email and calendars across Microsoft 365. Overloaded servers dropped out across regions, which piled more pressure on the ones still running. Early attempts to revive servers and shift traffic did not work. Microsoft ruled out a cyberattack and recovered by adding virtual machines, applying patches and rolling back configuration changes.

What are bulkheads in software?

The term comes from ship design. Bulkheads split a system into isolated sections so a failure in one stays there. In Nadella's example, a broken server or network gateway should not stop other servers and regions from handling Outlook traffic, and automatic failover should not carry the fault into healthy systems.

Nadella also pointed to a side effect of AI. It helps Microsoft find and fix bugs faster, he said, but every fix has to be patched in, "and that creates instability."

The outage reached users in India, the US, Canada, the UK, Australia, Japan, Singapore, Taiwan and New Zealand. Airlines, banks and large US companies were hit, and India's IndiGo and Akasa Air reported problems with their email support.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:microsoftoutlookoutagesatya nadellaexchange onlinemicrosoft 365

More from this day

See all →

More in: Software

See all →