Summary

On Monday, September 18th, we experienced 500 errors from images using the Amplience Virtual Staging Environment endpoint.

Root Cause

We discovered failures after a renewed certificate used on multiple services failed. One of the our internal services did not have an up to date root level certificate and was therefore unable to validate against the latest certificate change. This in turn caused the service to fail. Once we detected failures we begun the process of rolling back to the previous certificate and we are now in the process of working towards a permanent fix.

Corrective and Preventive Measures

Internally, we are working towards a permanent fix going forward.

Posted Sep 19, 2023 - 17:21 BST

Resolved

We have monitored traffic and error logs. We can confirm VSE is operational. We do apologise for any inconveniences this may have caused.

Posted Sep 18, 2023 - 20:14 BST

Monitoring

Our Engineering team have rolled back a change which you should now see vse working as expected

Posted Sep 18, 2023 - 19:52 BST

Identified

We have identified a failure with VSE. Our Operations and Engineering teams are currently investigating the cause and looking to fix the problem

Posted Sep 18, 2023 - 19:45 BST

This incident affected: Dynamic Content.