Incident Summary
Between approximately July 13 and July 14, 2026, some customers were intermittently unable to send offer letters, and some document/resume previews failed to generate. While source files were uploaded successfully, the conversion step required for previews and offer generation failed. No data loss occurred.

Detection
The issue was identified on July 14, 2026, at approximately 10:43 AM ET, following reports from customers to our Support team. Engineering confirmed the impact through monitoring, which showed a steady increase in process counts within the document-converter service until it reached a hard system limit, causing subsequent tasks to fail.

Root Cause
The incident was caused by a "process leak" introduced during a recent platform upgrade. A change in how the system handles document conversions meant that each conversion started a new background process that was never properly closed. Over several days, these background processes accumulated until the system reached its maximum capacity. Once this limit was hit, the service could no longer start the new processes required to generate offer letters or document previews.

Resolution

• Initial Mitigation: At 11:20 AM ET on July 14, engineers restarted the affected service pods. This cleared the accumulated background processes and immediately restored the ability for customers to send offer letters and view previews.
• Permanent Fix: A code update was developed and deployed at 8:43 PM ET on July 14. This fix ensures that all background processes are correctly terminated and cleaned up after every conversion, preventing any future accumulation.

Preventative Measures

• Code Improvements: Implemented a robust process-management system (using a "reaper" process) to ensure all background tasks are automatically closed upon completion.
• Enhanced Monitoring: New automated alerts have been configured to notify the engineering team if process usage exceeds 50% of the system limit. This will allow us to identify and resolve similar resource leaks days before they impact customers.
• Backlog Processing: A separate initiative was completed to identify and reprocess any document or resume previews that failed to generate during the outage window to ensure all previews are available to users.