Microsoft Azure has now recovered operations as usual and our monitoring did not detect any issues.
We are now expecting a full root cause analysis from Microsoft Azure and we will discuss internally how we can improve our resilience to such issues.
The whole Cloud team would like to thank you for your understanding.
---------- Here is Azure's statement Summary of Impact: Between 07:44 UTC and 13:10 UTC on 18 Jan 2023, a subset of customers using Storage in West Europe may have experienced higher than expected latency, timeouts or HTTP 500 errors when accessing data stored on Storage accounts hosted in this region. Azure services with dependencies on storage may also have experienced impact, such as Virtual Machines.
Posted Jan 18, 2023 - 13:42 UTC
Monitoring
Dear customers,
On the 18th of January 2023, from 07:40 UTC, customers may have experienced failure in querying labels or stores' statuses, via API or via our user interface, VUSION Manager. At 8:35 UTC, we have switched to our secondary unit to mitigate most of the issues.
The root cause is a Microsoft Azure outage on Storage. Here is Microsoft statement on https://status.azure.com:
"Warning Storage - West Europe - Investigating
Impact Statement: Starting at 07:44 UTC on 18 Jan 2023, a subset of customers using Storage in West Europe may experience higher than expected latency, timeouts or HTTP 500 errors when accessing data stored on Storage accounts hosted in this region. Additional services with dependencies on storage may also experience impact.
Current Status: We have determined that a subset of storage clusters in one of the region datacenter experienced a drop in availability and may have rebooted. We are seeing signs of recovery and are working through the recovery of the impacted clusters and services. An update will be provided within 60 minutes, or as events warrant.
This message was last updated at 09:29 UTC on 18 January 2023."
Our Cloud Engineer keep monitoring the platform behaviour and the other impacts this outage may have.
On behalf of SES-imagotag, the whole Cloud team would like to apology for the incident.
Posted Jan 18, 2023 - 10:26 UTC
This incident affected: Europe (VUSION Manager - Europe, VUSION Cloud API - Europe).