Postmortem -
Read details
Apr 16, 18:14 EDT
Resolved -
The incident has been fully resolved. Regular service has been restored, and all systems are operating as expected.
Impact:
Users should no longer experience any issues related to this incident.
If you are still experiencing issues, please do not hesitate to reach out to the support team and update your ticket or report any problems you haven't reported yet.
Service has been fully restored. We apologize for the degradation in services. We thank you for your understanding. If you continue to experience issues, please don't hesitate to contact our support team.
We will post an RCA after an internal investigation.
Apr 13, 22:33 EDT
Update -
We are continuing to rollback collector versions on each cluster.
Impact:
Customers may continue to experience collector SNMP monitoring issues
We will be having cluster maintenance windows as we roll back the collector upgrade on each cluster.
We are rolling back the remaining clusters currently.
Please report any related issues to Auvik Support so we can track and assist further.
Next Steps:
We are implementing mitigation measures and will provide progress updates.
Apr 13, 21:55 EDT
Identified -
Our team has identified a suspected cause of the Collector SNMP monitoring issue and is taking steps to remediate.
Impact:
Customers may continue to experience collector SNMP monitoring issues
We will be having cluster maintenance windows as we roll back the collector upgrade on each cluster.
We are rolling back the EU2 cluster currently.
Please report any related issues to Auvik Support so we can track and assist further.
Next Steps:
We are implementing mitigation measures and will provide progress updates.
Apr 13, 21:10 EDT
Investigating -
We are currently investigating reports of a collector affecting monitoring.
Impact:
Customers may experience issues with the SNMP monitoring protocol.
Other services are not affected.
Auvik will be performing an emergency rollback of the collector upgrade this weekend.
Next Steps:
Our team is working to identify contributing factors. Updates will follow as more information becomes available.
Apr 13, 20:32 EDT