Assets failing to load in content Hub
Incident Report for Amplience
Postmortem

Issue summary

On the afternoon of Tuesday 25th July, we identified an issue with loading assets in Content Hub.

Root Causes

We identified shard failures which contributed to the loss of asset visibility from the Content Hub Interface. This also affected uploading new assets and updating due to the loss of visibility into existing and new assets.‌

Corrective actions

Internally, we have conducted discussions on how to enhance our ability to mitigate such a failure from occurring again. New research and development will be conducted around existing systems as well as looking at new methods to avoid disruption to live and existing content within the platform.

Posted Jul 31, 2023 - 15:40 BST

Resolved
We have rolled out a change to mitigate the failed shards, we are now seeing stability across Content Hub and will continue to monitor this evening.
Posted Jul 25, 2023 - 18:19 BST
Monitoring
We identified a second failure which will directly affect Assets loading in Content Hub. We will continue to monitor as shards recover.
Posted Jul 25, 2023 - 15:41 BST
Identified
We have identified a second failure which will directly affect Assets loading in Content Hub. We will continue to work on a secondary fix.
Posted Jul 25, 2023 - 15:35 BST
Monitoring
We have identified the issue and a fix applied. We can see assets are becoming available again and we will continue to monitor the situation.
Posted Jul 25, 2023 - 15:19 BST
Update
We are currently investigating a failed shard affecting assets loading on Content Hub
Posted Jul 25, 2023 - 15:01 BST
Investigating
We are currently investigating a failed shard affecting assets loading on Content Hub
Posted Jul 25, 2023 - 15:01 BST
This incident affected: Content Hub.