Lever -hosted job site down

Incident RCA: Job Site Outage

Incident Date: October 3, 2025
Start Time: 4:51 PM ET
End Time: 6:00 PM ET

Impact Summary

Between 4:51 PM and 6:00 PM ET on October 3, 2025, users experienced degraded performance and eventual unavailability across multiple job boards.

Root Cause

The outage was triggered by an unexpected pattern of incoming requests that, while not unusually high in volume, each initiated resource-intensive queries. This combination of request frequency and prolonged query execution overwhelmed backend systems, leading to degraded performance and service unavailability. The traffic resembled automated crawling behavior, which our systems were not optimized to handle at that level of query complexity and concurrency.

Resolution

To reduce server load and restore service availability, the impacted job site was temporarily disabled. This action immediately alleviated pressure on backend systems and allowed all job boards to recover. The site remained offline over the weekend and was re-enabled on October 6, 2025, after Engineering confirmed that the irregular traffic had subsided.

Preventive Actions

To prevent similar incidents in the future, the following measures have been implemented:

• Enhanced Caching: Improvements to our job site caching mechanisms now limit the ability of similar request patterns to generate excessive server queries.
• Updated Crawling Guidelines: We have revised our web crawling directives to encourage automated crawlers to reduce the frequency of their requests. This change helps maintain system stability by minimizing the impact of high-volume automated traffic from compliant sources.