Fix OpenAI API Rate Limit Errors in Node.js

Your Node.js backend calls the OpenAI API a few dozen times an hour in development and everything works fine. Then you ship it, real users show up, and suddenly requests start failing with 429 Too Many Requests and a body that says rate_limit_exceeded. If you're reading this because that error just showed up in your logs, here's the actual fix — not just an explanation of what a rate limit is.

The OpenAI API enforces limits on two axes: requests per minute (RPM) and tokens per minute (TPM), and the specific numbers depend on your usage tier and the model you're calling. Hitting either ceiling returns the same 429 status, and the fix in both cases is the same pattern: back off, retry with the right delay, and stop firing requests faster than your tier allows in the first place.

Why the 429 error happens in production but not locally

Locally, you're the only caller, making requests one at a time with pauses between them while you read the output. In production, a burst of concurrent user requests can each trigger an OpenAI call within the same second — a chat feature under load, a batch job processing a queue, or a webhook handler retrying itself. OpenAI counts requests and tokens per minute across your entire account, not per user, so ten simultaneous users can trip a limit that felt generous during solo testing.

The fix isn't "upgrade your tier and hope" — even paid, high-tier accounts hit rate limits under bursty traffic, because limits are a ceiling on burst rate, not a promise of unlimited throughput. You need retry logic regardless of tier.

Reading the error correctly

When the OpenAI Node.js SDK throws on a 429, the error object carries the information you need to retry correctly:

try {
  const response = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: prompt }],
  });
} catch (err) {
  if (err.status === 429) {
    const retryAfter = err.headers?.["retry-after"];
    console.log("Rate limited. Retry after (seconds):", retryAfter);
  }
}

The retry-after header, when present, tells you exactly how long to wait before the next attempt — use it instead of guessing. If it's missing, fall back to exponential backoff.

Implementing exponential backoff with jitter

A naive retry loop that immediately retries makes the problem worse — every failed request becomes two requests firing at nearly the same moment. Exponential backoff with jitter spaces retries out and spreads concurrent callers across different delays so they don't all retry in lockstep:

async function callWithRetry(fn, { maxRetries = 5, baseDelayMs = 1000 } = {}) {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (err) {
      const isRateLimit = err.status === 429;
      const isLastAttempt = attempt === maxRetries;

      if (!isRateLimit || isLastAttempt) throw err;

      const retryAfterHeader = err.headers?.["retry-after"];
      const backoffMs = retryAfterHeader
        ? Number(retryAfterHeader) * 1000
        : baseDelayMs * 2 ** attempt;

      const jitter = Math.random() * 300;
      await new Promise((resolve) => setTimeout(resolve, backoffMs + jitter));
    }
  }
}

// Usage
const completion = await callWithRetry(() =>
  openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: prompt }],
  })
);

This handles the immediate failure, but it doesn't stop you from generating the bursts that cause 429s in the first place. For that, you need to control concurrency upstream.

Queueing requests instead of firing them all at once

If your endpoint processes a batch — summarizing 50 documents, embedding a list of records — don't Promise.all() fifty OpenAI calls simultaneously. Use a concurrency limiter so only a fixed number of requests are in flight at any moment:

import pLimit from "p-limit";

const limit = pLimit(5); // max 5 concurrent OpenAI calls

const results = await Promise.all(
  documents.map((doc) =>
    limit(() => callWithRetry(() => summarize(doc)))
  )
);

Five concurrent requests, each with retry-and-backoff behind it, is far less likely to trip your per-minute ceiling than fifty requests fired at once. Tune the concurrency number against your account's actual RPM/TPM limits, visible in your OpenAI dashboard's usage/limits page.

Common mistakes that make rate limiting worse

• Retrying immediately with no delay, which turns one 429 into a retry storm
• Not reading the retry-after header and always using a fixed backoff, even when OpenAI told you the exact wait time
• Retrying non-retryable errors the same way — a 400 (bad request) or 401 (auth) will never succeed on retry, so only retry on 429 and 5xx
• Running unbounded concurrency in batch jobs, which is the single most common cause of hitting a TPM ceiling in production
• Not logging rate-limit events at all, so the team doesn't notice the app is silently swallowing failed generations for users

Best practices for production

1. Wrap every OpenAI call in retry-with-backoff logic, not just the ones you've personally seen fail.
2. Cap concurrency with a limiter (p-limit or a small custom semaphore) around any batch or bulk operation.
3. Log every 429 with the endpoint, retry count, and eventual outcome so you can see the pattern, not just the individual failure.
4. If your app depends on a hard SLA, consider a request queue (BullMQ, a simple in-memory queue for low volume) that throttles outbound calls to a rate you know is under your tier's ceiling, rather than reacting to 429s after the fact.
5. Check your usage tier's actual RPM/TPM numbers rather than assuming — they differ by model and by how much you've spent historically.

Frequently Asked Questions

Does upgrading my OpenAI usage tier fix rate limit errors permanently? It raises the ceiling, but it doesn't remove the need for retry logic — any tier can be hit by a sharp enough burst of concurrent traffic, so backoff-and-retry should be in place regardless of tier.

Should I retry on every error type, not just 429? No. Retry on 429 and 5xx server errors, since those are transient. A 400 or 401 will fail identically on every retry and just wastes time and API calls.

What's a safe concurrency limit for batch OpenAI calls in Node.js? There's no universal number — it depends on your tier's RPM/TPM and the size of your prompts. Start around 3–5 concurrent requests for a typical paid tier, watch your OpenAI dashboard's usage graph, and tune from there.

Does the retry-after header always appear on a 429 response? Not always — OpenAI includes it when it can compute an exact reset time, but if it's absent, exponential backoff with jitter is the correct fallback rather than guessing a fixed delay.

Key Takeaways

A 429 from the OpenAI API is not a bug in your code — it's the API telling you to slow down and telling you, via retry-after when available, exactly how long to wait. The fix is retry logic with exponential backoff and jitter on every call, plus a concurrency limiter around any batch operation, so your app degrades gracefully under load instead of dropping user-facing generations. Add the callWithRetry wrapper and a p-limit gate around your OpenAI calls today, and rate limit errors stop being a production incident and become a handled edge case.