A year on from the AWS outage: what a DNS fault taught data teams about hidden dependencies
A year ago this week, a fault in one AWS region stopped a long list of apps, from banking and messaging to games and smart-home gadgets, for hours. Most of the people affected had never heard of the service that failed. That is the lesson for anyone who runs reports on live business data: the thing that breaks is rarely the thing you chose, and it is often the thing nobody wrote down.
Trinetro Labs · Published 25 October 2026 · 4 min read
What happened on 20 October 2025
The problem began in US-EAST-1, one of the busiest AWS regions, in the late hours of 19 October on the US Pacific clock, which was 20 October for most of the world. AWS later explained, in its own post-event summary, that a latent race condition in the automated DNS management for DynamoDB, its database service, left the regional DynamoDB address unresolvable. Programs that asked where to find DynamoDB got no usable answer. It was not a cyberattack. It was a flaw in how DNS records were managed, one that had been waiting for the right timing. AWS fixed the DNS problem within hours, but recovery was not instant. Other internal systems that depend on DynamoDB, including the one that launches new EC2 servers and the health checks behind its network load balancers, were impaired next, and AWS reported all services back to normal later that day.
Why one fault reached so far
Three things made a single DNS problem so large:
- Concentration. A very large number of companies run something important in the same region, often because it was the default.
- Chains of dependency. DynamoDB sat underneath other AWS services, and those sat underneath customers' apps. A failure at the bottom travels upwards through every layer that quietly relies on it.
- Slow recovery after the fix. Restoring one service did not restore the services that had stalled while waiting for it. Queues had to drain and systems had to be brought back in the right order.
None of this is unique to AWS. Every large cloud has had its own version of this day, and the useful question is not who to blame but what your own setup would have done.
What it means for the numbers you report from
A reporting stack has the same hidden chains. A finance dashboard looks like one thing, but behind it sit a warehouse, a scheduler that refreshes it, a sign-in service, an email service that delivers the Monday pack, and often a vendor's API for exchange rates or payments. If any one of them stalls, the symptom you see is usually a stale number, not an error message. A stale number that looks fresh is worse than a blank page. The risk is not only downtime. It is the quiet case where data stops arriving, the dashboard keeps showing yesterday's total, and a decision is made on it. A good report says when it was last refreshed, and a good team notices when that time stops moving.
Six checks to run this quarter
- List every service between your source data and the report your leadership reads. Include sign-in, scheduling, storage and email delivery.
- Write down which cloud region each one runs in. If the answer for everything is the same region, you have found your concentration risk.
- Make sure every report shows its last refresh time, and decide who looks at it when it is old.
- Ask each vendor for its status page and its past incident reports. A vendor that publishes plain, detailed ones is easier to trust than one that does not.
- Decide what the business does when the dashboard is unavailable for half a day. If the answer is "wait", write that down as a decision, not an accident.
- Keep an export of the figures that close the books or go to the board, so a month-end does not depend on a single live system.
If you are reviewing how your analytics tools handle access and failure, the data access control checklist and our post on how to read a cloud incident report are good next steps.
Frequently asked questions
What caused the AWS outage of 20 October 2025?
AWS attributed it to a latent race condition in the automated DNS management for DynamoDB in the US-EAST-1 region, which left the regional DynamoDB address unresolvable. It said it was not a cyberattack. Other systems that depend on DynamoDB were impaired as it recovered.
Does using a different cloud provider protect my reports?
Not by itself. Every large provider has had regional incidents. What helps is knowing your dependencies, spreading critical ones across regions where it is practical, showing refresh times on every report, and keeping exports of the figures that matter most.
How can I tell if a dashboard is showing stale data?
Look for a last-refreshed time on the report itself. If the number has not moved when you would expect it to, check the data refresh before you act on it.
Where can I read AWS's own account of the incident?
AWS published a post-event summary for the event, linked from this article. It is the primary source for the cause and the timeline.