Authentication Issue - 500 Error and Email/Calendar Sync Failures

Incident Report for Lever

Postmortem

Incident Summary

Between July 2, 2026, and July 8, 2026, Some customers experienced intermittent failures affecting email and calendar services. The primary impacts included errors when sending or syncing emails and occasional "500 errors" during login. These issues were caused by a technical regression in the underlying runtime environment (Node.js) that handles outbound connections to third-party services.

For a subset of customers using specific email integration settings, the connection instability caused the system to incorrectly perceive a successful email send as a failure. This triggered an automated retry mechanism, which in some cases resulted in a "retry loop" where the same communication was re-sent to recipients at regular intervals until the connection was stabilized.

Detection

The issue was identified following reports of intermittent service disruptions. Our support and engineering teams conducted a thorough investigation into these reports, identifying a specific pattern of connection failures within our system logs. Further analysis confirmed that the impact began on approximately July 2, 2026, following a routine update to our service infrastructure.

Root Cause

The root cause was a defect in a specific version of Node.js (version 24.17.0), the software environment used to run our services. This version contained a flaw in how it managed "keep-alive" connections—reusable connections used to communicate with external services. Under certain conditions, the software would prematurely close these connections before a task was completed, resulting in failed email sends and data sync errors.

Resolution

To resolve the issue, our engineering team performed the following steps:

  • Identification: Correlated the "Premature close" errors to a known regression in the Node.js runtime.
  • Upgrade: Updated our base system images to Node.js version 24.18.0, which contains the official fix for this connection issue.
  • Deployment: Rebuilt and redeployed critical services (including email and calendar components) on the corrected version.
  • Verification: Confirmed that connection errors ceased and service stability returned to normal levels on July 8, 2026.

Preventative Measures

To prevent similar issues in the future, we are implementing the following improvements:

  • Version Pinning: We have updated our build process to ensure we deploy only versions that have been fully verified.
  • Enhanced Monitoring: We are adding specific alerts for outbound connection errors to detect and respond to similar communication failures more rapidly.
  • Automated Testing: We are implementing new synthetic checks for email and calendar synchronization to proactively identify issues before they impact customers.
  • Improved Visibility: We are updating our internal dashboards to provide better visibility into the exact software versions running in our production environment to speed up future diagnoses.
Posted Jul 09, 2026 - 13:43 PDT

Resolved

This incident is resolved. A Root Cause Analysis (RCA) will be published to this issue within 7-10 business days.
Posted Jul 09, 2026 - 09:19 PDT

Monitoring

A fix has been implemented as of approximately 12:15pm PST/PDT on 7/8/2026. Email and Calendar sync are now fully functional for Lever instances configured with Nylas authentication.

We are continuing to monitor the effectiveness of the fix. A Root Cause Analysis (RCA) will be published to this issue within the standard Service Level Agreement.
Posted Jul 08, 2026 - 13:03 PDT

Identified

On 7-3-2026 we were made aware of an issue impacting Lever instances with Nylas authentication (generally used for customers with on-premise Microsoft Exchange email/calendar services). Only Nylas authentication is impacted, there has been no similar behavior observed for customers authenticating with Office 365 or Google services.

Users may experience a "500 error" message when attempting to reset email/calendar permissions and we have observed several unexpected email loops causing candidates to receive duplicate emails every 10 to 15 minutes as a result of this issue.

Root cause: The 500s and email failures are caused by a known bug in Node.js v24.17.0 (https://github.com/nodejs/node/issues/63989). The v24.17.0 security release (CVE-2026-48931) changed `http.Agent` keep-alive socket handling so that `node-fetch` throws `ERR_STREAM_PREMATURE_CLOSE` ("Premature close") on gzipped responses to external providers (Nylas/Google). Our services picked it up this bug via the Node 18→24 migration (Occurred on 06-30-2026). The bug was fixed upstream in Node 24.18.0

This issue should be fixed in a deployment today (7-8-2026). We will provide updates on the issue status and deployment timeline as soon as possible.
Posted Jul 08, 2026 - 10:17 PDT
This incident affected: Third-Party Workflow Dependencies (Microsoft Exchange (Nylas) Email/Calendar Integration).