Resumes and Offers Failing to Load

Incident Report for Lever

Postmortem

We understand how important this functionality is to your hiring workflows, and our team moved quickly to investigate and address the issue. We can confirm this issue has been fully resolved, and our team has taken steps to ensure system stability moving forward. 

Customer Impact

  • On January 6th, 2026, between 15:07 and 18:20 (UTC), users were unable to upload resumes or documents in Lever ATS.
  • Upload attempts failed with an authorization-related error.
  • Any new uploads during the incident window may result in references to non-existent files within the ATS. Unfortunately, these files are not recoverable.
  • Other areas of the application continued to function normally.

Root Cause

The incident was caused by an updated configuration in Amazon S3 access policies following a security-related infrastructure change.

As part of a security initiative, an existing IAM policy condition was updated in accordance with an AWS-recommended best practice. While this change was intended to improve security, the revised policy did not behave as expected in our environment, resulting in S3 denying permission for file uploads.

Because file uploads rely on this S3 permission, customers encountered authorization errors when uploading resumes and documents.

Resolution

Once the issue was identified, the infrastructure team reverted the policy change to the previously functioning configuration.

After the rollback:

  • File uploads resumed successfully
  • Error rates returned to normal levels
  • Customer impact was fully resolved

The system was verified as restored by confirming:

  • Successful resume uploads from the application
  • Elimination of S3 authorization errors in logs and dashboards

Invalid file reference cleanup

  • Our Engineering teams are in the process of removing any invalid references to files that were not successfully uploaded during this time

Preventative Actions

To reduce the likelihood of similar issues in the future, we are implementing the following improvements:

  1. Post-Deployment Monitoring
* Enhanced monitoring after infrastructure and security-related changes in production
  1. Automated Alerts
* Alerts for elevated file-upload failures and S3 authorization errors to reduce detection time
  1. Change Validation
* Additional validation and testing of IAM policy changes, especially those involving AWS-recommended updates
  1. Documentation & Review
* Improved documentation and cross-team review for security-driven infrastructure changes

These steps will help detect issues sooner and minimize customer impact in the event of future changes.

Posted Jan 21, 2026 - 07:27 PST

Resolved

This incident has been resolved.
Posted Jan 06, 2026 - 05:51 PST

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Jan 06, 2026 - 04:47 PST

Investigating

We are investigating an issue where resumes are not loading, offers are not displaying, and errors occur when manually adding resumes to opportunities. Our team is actively working to identify the root cause and restore normal functionality.
Posted Jan 06, 2026 - 02:52 PST
This incident affected: Global Data Center - LeverTRM (Hire).