Discovered: Nov 15, 2021 - UTC 19:38
Resolved: Nov 15, 2021 - UTC 20:05
The upgrade of database service hung during the automated process.
The refreshed TrafficInsights data was delayed for 25 minutes for Auvik Performance clients in the EU2 cluster. There was no data loss and no other services were affected.
11/15/2021 All times in UTC
19:38 - UTC Planned and pre-tested zero-downtime upgrade of backend service begins.
19:41 - UTC The service disconnects during the upgrade process.
19:43 - UTC Alerts are registered in Auvik’s monitoring.
19:58 - UTC Auvik engineering team begins investigation.
20:03 - UTC The automated upgrade completes, reconnecting the disconnected service with the platform.
20:05 - UTC Errors for the incident cease.
Auvik will review it’s testing processes and procedures to validate accuracy and completeness for zero-downtime upgrades as compared to actual production implementation.