Event Details:

On October 20th, 2020 between 12:21 UTC and 16:15 UTC, the Area 1 Security API service was inaccessible and not able to service requests.

Root Cause & Actions

An incorrect configuration of the API service was deployed that limited the available memory to the service causing it to terminate. Our monitoring of service health was not functioning properly allowing this condition to remain undetected for several hours.

The Area 1 engineering team corrected the configuration issue, restoring the operation of the service. A thorough review of the monitoring system was done to correct the lack of visibility and additional safeguards have been implemented to avoid this type of issue occurring again.