500 errors in the web portal and admin console
Incident Report for UserVoice
Postmortem

On February 24th from 6:03PM to 6:24PM EST, 43% of requests to UserVoice failed, and resulted in 500 errors for admins and end users.

Business Impact

  • Admins could have seen 500 errors when loading the admin console or performing actions within UserVoice
  • End users could have seen errors when trying to interact with the widget or web portal.

Root Cause

  • When adding an index to our live sql servers to improve our suggestion query performance, a mistake was made which caused the servers to be unresponsive while creating the new index.

What we are Doing to Prevent This

  • Our engineering team has identified process improvements that will prevent a similar mistake going forward.

We are sorry for the disruption this caused to your day. Please, let me know if you have any questions or concerns.

Claire Talbott

Support Manager

claire.talbott@uservoice.com

Posted Feb 26, 2018 - 16:57 EST

Resolved
This incident has been resolved.
Posted Feb 24, 2018 - 21:42 EST
Monitoring
The problem has been resolved, we will continue to monitor
Posted Feb 24, 2018 - 18:24 EST
Identified
We identified a problem with our database servers which is causing errors across the site. We will update you shortly
Posted Feb 24, 2018 - 18:09 EST