On August 5, 2019, from 5:49PM to 6:23PM EDT, the admin console and UserVoice API were down.

Business Impact

• Admins saw 503 errors in the admin console, or noticed excessive slowness when loading parts of the application.
• The UserVoice API returned errors.
• Users on the web portal or with the widgets may have briefly seen errors or slowness at the start of the incident, but only for a few moments.

Root Cause

• A configuration change during a routine Kubernetes Cluster upgrade caused an unexpected restart for key operational services. One of these services had trouble recovering, which caused the downtime.

What we are Doing to Prevent This

• Our team identified and fixed the issue that caused the trouble during restart.
• Our team also identified the setting that triggered the unplanned update, and how best to ensure that does not happen again.

We do apologize for the downtime. This interrupted your usage of UserVoice, and for that we truly are sorry.

If you have any questions or concerns, please, don’t hesitate to reach out!

Claire Talbott

Support Manager

claire.talbott@uservoice.com