Resumo
Between 12:20 PM and 1:55 PM PST, we experienced elevated DNS resolution errors for CloudFront distributions served from a subset of edge locations globally. During this time, customers may have received NXDOMAIN responses. Engineers were automatically engaged, immediately began investigating in multiple parallel paths and mitigated errors by taking the affected fleet of DNS servers out of service. Additionally, between 12:20 PM and 8:08 PM PST, we experienced longer than usual propagation times for changes to CloudFront configurations. The propagation delay was limited to certain types of changes, such as creating/deleting distributions, or updating the distributions for DNS or TLS certificate related changes. Following mitigation of the DNS resolution errors at 1:55 PM, end-user requests for content delivery from our edge locations were not affected and are being served normally. Invalidations operated normally throughout the event. Both issues have been resolved and all services are operating normally. We are confident this issue will not reoccur.