Intermittent request timeouts
Incident Report for UserVoice
Postmortem

On 4/20/17 from 1:53PM to 2:02PM EDT, 87% of requests to UserVoice failed.

Business Impact

During the incident, the following parts of the app were affected, and failed to load or gave an error 82% of the time.

  • Admin console
  • Web Portal (front end of your UserVoice site)
  • API
  • Mobile SDKs
  • Contributor Sidebar

Email was delayed, but no email was lost.

Root Cause

Our engineering team was doing a data operation that was too aggressive. We immediately killed the operation, which resolved the issue.

What we are Doing to Prevent This

  • We’ve identified the problem with the job that caused the issue, and have implemented a fix. Our engineering team is working on better QA prior to operations like this to prevent something similar from occurring in the future.

  • We have also added tooling to improve our ability respond to misbehaving jobs.

Downtime for any reason is something we take seriously. We apologize for the disruption this caused for you and your team. If you have any questions, you can reach us at support@uservoice.com.

Claire Talbott Support Manager

Posted Apr 21, 2017 - 17:59 EDT

Resolved
This incident has been resolved.
Posted Apr 20, 2017 - 16:41 EDT
Monitoring
Some increased database load was causing slow responses and timeouts from UserVoice, including the admin console and APIs. We've tracked down the issue and the site is functioning normally now.
Posted Apr 20, 2017 - 14:09 EDT