At 14:14 ET on July 27, 2021 our systems reported an increase in latency to our primary database cluster due to an abnormal and significant increase in traffic. This increase outpaced our scaling capabilities, and resulted in a cascading failure that brought down the database cluster responsible for our Hub Platform.
Recovery efforts were initiated quickly once the cause was identified, however a complication during recovery extended the outage.
By 16:19 ET the Hub Platform was back online, and all systems were brought back to normal operation shortly thereafter.
Remediation efforts on our infrastructure and product performance are underway to prevent a recurrence of this issue.