DNS Issue impacting multiple services

Incident Report for Amplience

Postmortem

Incident Summary

Duration: 36 minutes (22/07/2021 16.40 BST – 22/07/2021 17.16 BST)

On the afternoon of Thursday 22nd, July 2021, an Amplience supplier; Akamai experienced a Global CDN Incident. This impacted the following Amplience Services:

• Content Delivery
• Content Delivery v2
• Dynamic Imaging

Amplience Edge CDN’s are balanced between Akamai and Cloudflare to provide the best performance. Amplience Engineers redirected traffic from Akamai to Cloudflare and scaled up our servers to absorb the traffic. Once we were confident that Akamai had resolved their Incident, we reverted traffic back.

Root Cause

Akamai a global content delivery network, cybersecurity, and cloud service company experienced a disruption caused by a change in configuration and was then rolled back.

Corrective actions

• Review our auto failover rules so that we can redirect traffic quicker

If you wish to discuss this further, please reach out to your Customer Success Manager (csteam@amplience.com)

Posted Jul 26, 2021 - 15:00 BST

Resolved

This incident has been resolved.

Posted Jul 22, 2021 - 18:18 BST

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Jul 22, 2021 - 17:46 BST

Identified

We have identified that our provider; Akamai is having major network issues which is impacting a number of our services. We are working to mitigate this and will update you shortly.

Posted Jul 22, 2021 - 17:10 BST

Investigating

We have identified an issue and we are currently investigating.

Posted Jul 22, 2021 - 16:54 BST

This incident affected: Content Hub, Dynamic Content, and Dynamic Media.