Which errors should you actually retry?

Retry 408, 425, 429, 500, 502, 503 and 504; never retry 400, 401, 403, 404 or 422. The test is one question: could this exact request succeed if I sent it again unchanged? A 500 could. A 422 could not — the payload is wrong, and it will still be wrong in two seconds.

That covers the easy half. The half that causes outages is the other one: the request that got no response at all, and the POST that may already have gone through.

The short answer

• Retry a status code only when the server's answer means "not now": 408, 425, 429, 500, 502, 503, 504. A 4xx that describes the request itself — 400, 401, 403, 404, 422 — will fail identically forever.
• A retryable status is not the same as a status that is safe for a bare POST. 408, 425, 429 and 503 mean the server did no work. 500, 502 and 504 mean it might have.
• When 429 or 503 carries Retry-After, obey it. In the demo below, a client that read the header sent 2 requests; a client with its own 50 ms backoff sent 20 and finished no sooner.
• For network errors, error.syscall is the field that matters. connect and getaddrinfo mean the request never reached the application, so it is safe to resend even a POST. No syscall, or read/write, means it might have been processed.
• A timeout on a POST is not a failure — it is an unknown. Retrying a bare POST that timed out charged the card twice in the demo below. The same POST with an Idempotency-Key charged it once.

The decision table

Every code below was produced and observed on Node 23.5.0, against servers on localhost. Nothing here is from memory.

Status codes

Network errors, as Node reports them

Should you retry a 500?

Yes, but not blindly, and this is where most retry code is subtly wrong.

A 500 is a response. The server was reached, it ran your request, and something broke. What you do not know is where it broke. If the handler wrote a row, then threw while sending the email, you have a row. Retry the request and you get a second one.

So 500, 502 and 504 belong in a different class from 408, 425, 429 and 503. The second group are all statements by the server that it did not do the work: it never got the full request, it refused early data, it throttled you, it is not accepting traffic. Those are unambiguously safe. The first group are statements that something went wrong somewhere, which is not the same thing.

504 is the sharpest example. A gateway timeout means the proxy stopped waiting. The upstream service usually has not stopped working — it finishes the job thirty seconds later, into a socket nobody is reading. Retrying a 504 on a POST is a reliable way to do the same job twice.

Why 400, 404 and 422 are never worth retrying

Because the server is describing your request, not its own condition. A 422 says the JSON parsed and the content was rejected: a missing field, a date in the past, a currency it does not support. That is deterministic. Retrying it three times with exponential backoff turns one clear error into a four-second delay and the same clear error.

401 deserves a caveat. Do not retry the call; do refresh the token and issue a new call. Those look similar in a stack trace and are not the same operation — one is a retry loop, the other is a credential refresh with a single follow-up attempt. Code them separately, or a bad password becomes an infinite loop against someone's auth endpoint.

Should you obey Retry-After, or use your own backoff?

Obey it. The server knows when its window resets; your backoff is a guess.

The demo has an endpoint that returns 429 with Retry-After: 1 and starts succeeding one second after the first hit. Two clients, same work:

own backoff (50ms): 20 requests, 1033 ms
obey Retry-After  : 2 requests, 1004 ms

Identical latency, ten times the traffic (the blind count lands on 20 or 21 depending on the run). The client with its own backoff spent the whole second hammering an endpoint that had already told it exactly when to come back — and on a real API, that behaviour is what gets a key suspended. Retry-After also appears on 503 during deploys and maintenance, where the same logic holds. There is more on the sending and the receiving side of this in rate limits: respecting theirs, publishing yours.

One caution: Retry-After may be a delay in seconds or an HTTP date. Parse both, and cap whatever you get — a server that says Retry-After: 86400 should not put a worker to sleep for a day.

What do network errors actually look like in Node?

This is where retry logic goes wrong, because there is no status code to switch on and the error shapes are inconsistent. Reproduced against local servers — one that resets the connection, one that closes mid-upload, one with a full accept queue, and a hostname in the reserved .invalid TLD:

  label         code          syscall      GET   POST  message
  ECONNRESET    ECONNRESET    (none)       yes   no    "socket hang up"
  ECONNREFUSED  ECONNREFUSED  connect      yes   yes   "connect ECONNREFUSED 127.0.0.1:55594"
  ENOTFOUND     ENOTFOUND     getaddrinfo  no    no    "getaddrinfo ENOTFOUND no-such-host.invalid"
  EPIPE         EPIPE         write        yes   no    "write EPIPE"
  ETIMEDOUT     ETIMEDOUT     connect      yes   yes   "connect ETIMEDOUT 127.0.0.1:55593"  after 7818 ms

Three things in that output are worth keeping.

syscall is the useful field, not code. connect and getaddrinfo both mean the failure happened before any bytes of your request reached an application. That makes them safe to resend regardless of method — a POST that was never sent cannot have been processed twice. An error with no syscall, or with read or write, happened after the connection was established, and is ambiguous.

ECONNREFUSED is the safest error you can get. It is also the one people most often treat as fatal. Nothing was listening on the port. Resend it, including a POST — the process you were talking to is restarting, and it will be back.

ENOTFOUND is the only one of these worth failing fast on. DNS returned a definitive "no such name". Retrying re-asks a question that already has an answer. The exception is a service that was just registered, where the record genuinely does propagate — but then you are waiting for DNS, not retrying a request, and the timing is different.

fetch complicates this. Undici wraps everything in a generic error and hides the real code one level down:

reset    TypeError: "fetch failed"  cause=SocketError code=UND_ERR_SOCKET "other side closed"
refused  TypeError: "fetch failed"  cause=Error code=ECONNREFUSED "connect ECONNREFUSED 127.0.0.1:55594"
dns      TypeError: "fetch failed"  cause=Error code=ENOTFOUND "getaddrinfo ENOTFOUND no-such-host.invalid"

If your classifier reads err.code on a fetch failure it sees undefined for all three and retries none of them, or retries all of them. Read err.cause. Note that a reset arrives as UND_ERR_SOCKET, not ECONNRESET — a set of libc error codes is not enough to classify undici failures.

Is a timeout safe to retry?

For a GET, yes. For a POST, this is the case almost nobody codes correctly.

A timeout tells you that no response arrived. It tells you nothing about whether the server did the work. Those are two different worlds and the client cannot distinguish them — the same impossibility behind why exactly-once delivery is a lie.

The demo has a /charge endpoint that records the charge, then waits 900 ms before replying. The client gives up after 300 ms and retries once:

bare POST:
  attempt 1: TimeoutError — no answer, retrying
  attempt 2: TimeoutError — no answer, retrying
  charges applied: 2 ["ch_1","ch_2"]

POST with Idempotency-Key:
  attempt 1: TimeoutError — no answer, retrying
  attempt 2: HTTP 200 replayed ch_1
  charges applied: 1 ["ch_1"]

Two charges from one intent, and the client never saw a single successful response. Every log line it has says the operation failed.

The fix is not a longer timeout — that only widens the window before the same thing happens. It is a key the client generates before the first attempt and reuses on every retry, so the server can recognise the second call as the same call:

const key = crypto.randomUUID()          // generated ONCE, outside the loop
for (let attempt = 1; attempt <= 3; attempt++) {
  try {
    return await fetch(url, { method: 'POST', body, headers: { 'Idempotency-Key': key } })
  } catch (e) { await sleep(2 ** attempt * 100 * Math.random()) }
}

Generating the key inside the loop is the same bug with more ceremony. Note also that AbortSignal.timeout produces a TimeoutError — not ETIMEDOUT, which in Node means a TCP handshake that got no reply.

Does the HTTP method change the answer?

It changes it more than the error does. GET, HEAD, PUT and DELETE are idempotent: sending them twice has the same effect as sending them once. POST is not — each one is a new request to create something.

So the same ECONNRESET is safe after a GET and dangerous after a POST, and any retry helper that takes only an error and returns a boolean is missing half its inputs. It needs the method, and it needs to know whether an idempotency key was attached — because a keyed POST behaves like an idempotent request, which is the entire point of the key.

That gives the whole rule in a dozen lines:

const NEVER_PROCESSED = new Set([408, 425, 429, 503])   // server says it did no work
const MAYBE_PROCESSED = new Set([500, 502, 504])        // it might have
const RETRYABLE_ERR   = new Set(['ECONNRESET', 'ECONNREFUSED', 'ETIMEDOUT', 'EPIPE', 'EAI_AGAIN'])
const PRE_FLIGHT      = new Set(['connect', 'getaddrinfo'])  // never reached the app
const IDEMPOTENT      = new Set(['GET', 'HEAD', 'PUT', 'DELETE', 'OPTIONS', 'TRACE'])

function shouldRetry({ status, code, syscall, method = 'GET', idempotencyKey }) {
  const replayable = IDEMPOTENT.has(method) || Boolean(idempotencyKey)
  if (status !== undefined) {
    if (NEVER_PROCESSED.has(status)) return true
    if (MAYBE_PROCESSED.has(status)) return replayable
    return false
  }
  if (!RETRYABLE_ERR.has(code)) return false
  if (PRE_FLIGHT.has(syscall)) return true
  return replayable
}

Deciding whether to retry is this function. Deciding how — backoff, jitter, attempt limits, dead-lettering — is the other article. The two together are what a webhook sender needs; the receiving side of the same contract is in receive a webhook without losing events.

What I could not reproduce

EAI_AGAIN — the temporary DNS failure — is in the classifier above but not in the observed output. It comes from getaddrinfo, which uses the system resolver, and the only ways to make it fail temporarily are to change the machine's DNS settings or to take the network down. Pointing Node's c-ares resolver at a dead nameserver on localhost produces a different code, ETIMEOUT on syscall queryA, not EAI_AGAIN. It belongs in the retryable set on the same reasoning as the other pre-flight failures — no request was sent — but I did not observe it, so treat that row as reasoning rather than measurement.

ETIMEDOUT is observed, and reproducing it locally took a trick: a listener with backlog: 1 whose event loop is blocked never drains its accept queue, and macOS then silently drops further SYNs. The connection fails after 7.8 seconds with connect ETIMEDOUT. That figure is the OS handshake timeout, not something you set — which is why a connect timeout in your client is worth configuring.

Check it yourself

One file, no dependencies, no network beyond loopback. It runs all four demonstrations: the status table, Retry-After, the five network errors, and the double charge. It takes about 11 seconds, most of it waiting for ETIMEDOUT.

// retry-lab.mjs — Node 23. Loopback only, ports 55591-55595.
import http from 'node:http'
import net from 'node:net'
import { spawn } from 'node:child_process'

const sleep = ms => new Promise(r => setTimeout(r, ms))
const log = s => console.log(s)

// ------------------------------------------------------------- the classifier
const NEVER_PROCESSED = new Set([408, 425, 429, 503])   // server says it did no work
const MAYBE_PROCESSED = new Set([500, 502, 504])        // it might have
const RETRYABLE_ERR   = new Set(['ECONNRESET', 'ECONNREFUSED', 'ETIMEDOUT', 'EPIPE', 'EAI_AGAIN'])
const PRE_FLIGHT      = new Set(['connect', 'getaddrinfo'])  // failed before the app saw it
const IDEMPOTENT      = new Set(['GET', 'HEAD', 'PUT', 'DELETE', 'OPTIONS', 'TRACE'])

function shouldRetry({ status, code, syscall, method = 'GET', idempotencyKey }) {
  const replayable = IDEMPOTENT.has(method) || Boolean(idempotencyKey)
  if (status !== undefined) {
    if (NEVER_PROCESSED.has(status)) return true
    if (MAYBE_PROCESSED.has(status)) return replayable
    return false
  }
  if (!RETRYABLE_ERR.has(code)) return false
  if (PRE_FLIGHT.has(syscall)) return true
  return replayable
}

// ----------------------------------------------------------------- servers
let charges = []
const seenKeys = new Map()

const api = http.createServer(async (req, res) => {
  const url = new URL(req.url, 'http://x')

  if (url.pathname === '/status') {
    const code = Number(url.searchParams.get('code'))
    if (code === 429 || code === 503) res.setHeader('Retry-After', '1')
    res.writeHead(code); return res.end(String(code))
  }
  if (url.pathname === '/flaky') {                   // rate limited for one second
    api.hits++
    api.readyAt ??= Date.now() + 1000
    const wait = api.readyAt - Date.now()
    if (wait > 0) {
      res.setHeader('Retry-After', String(Math.ceil(wait / 1000)))
      res.writeHead(429); return res.end('slow down')
    }
    res.writeHead(200); return res.end('ok')
  }
  if (url.pathname === '/charge') {                  // works, then answers too late
    const key = req.headers['idempotency-key']
    if (key && seenKeys.has(key)) { res.writeHead(200); return res.end('replayed ' + seenKeys.get(key)) }
    const id = 'ch_' + (charges.length + 1)
    charges.push(id)
    if (key) seenKeys.set(key, id)
    await sleep(900)                                 // the acknowledgement is lost
    res.writeHead(200); return res.end(id)
  }
  res.writeHead(404); res.end('no')
})
api.on('connection', s => s.on('error', () => {}))   // aborted clients are the point here

const resetter = net.createServer(s => { s.on('error', () => {}); s.on('data', () => s.destroy()) })
const closer   = net.createServer(s => { s.on('error', () => {}); s.once('data', () => setTimeout(() => s.destroy(), 5)) })

await new Promise(r => api.listen(55591, '127.0.0.1', r))
await new Promise(r => resetter.listen(55592, '127.0.0.1', r))
await new Promise(r => closer.listen(55595, '127.0.0.1', r))

// -------------------------------------------------------------- 1. statuses
log('\n=== 1. status codes ===')
log('  code  retry  bare POST  Retry-After')
for (const code of [200, 400, 401, 403, 404, 408, 422, 425, 429, 500, 502, 503, 504]) {
  const r = await fetch(`http://127.0.0.1:55591/status?code=${code}`)
  const yn = b => (b ? 'yes' : 'no ')
  log(`  ${code}   ${yn(shouldRetry({ status: r.status }))}    ${yn(shouldRetry({ status: r.status, method: 'POST' }))}` +
      `        ${r.headers.get('retry-after') ?? '-'}`)
}

// ------------------------------------------------------- 2. obey Retry-After
log('\n=== 2. Retry-After vs your own backoff ===')
for (const obey of [false, true]) {
  api.hits = 0; api.readyAt = null
  const t0 = Date.now()
  for (let attempt = 1; attempt <= 40; attempt++) {
    const r = await fetch('http://127.0.0.1:55591/flaky')
    if (r.status === 200) break
    const after = Number(r.headers.get('retry-after')) * 1000
    await sleep(obey && after ? after : 50)
  }
  log(`  ${obey ? 'obey Retry-After  ' : 'own backoff (50ms)'}: ${api.hits} requests, ${Date.now() - t0} ms`)
}

// ------------------------------------------------------- 3. network errors
log('\n=== 3. network errors, as node:http reports them ===')
log('  label         code          syscall      GET   POST  message')
const show = (label, e, extra = '') => log(
  `  ${label.padEnd(13)} ${String(e.code).padEnd(13)} ${String(e.syscall ?? '(none)').padEnd(12)} ` +
  `${(shouldRetry({ code: e.code, syscall: e.syscall }) ? 'yes' : 'no ')}   ` +
  `${(shouldRetry({ code: e.code, syscall: e.syscall, method: 'POST' }) ? 'yes' : 'no ')}   "${e.message}"${extra}`)

const probe = (label, opts, bigBody) => new Promise(done => {
  const req = http.request(opts, r => { r.resume(); r.on('end', () => done(log(`  ${label} -> HTTP ${r.statusCode}`))) })
  req.on('error', e => done(show(label, e)))
  if (bigBody) {
    const b = Buffer.alloc(1 << 16, 65); let i = 0
    const pump = () => { while (i++ < 6000) if (!req.write(b)) return req.once('drain', pump); req.end() }
    pump()
  } else req.end()
})
await probe('ECONNRESET',   { host: '127.0.0.1', port: 55592, path: '/' })
await probe('ECONNREFUSED', { host: '127.0.0.1', port: 55594, path: '/' })
await probe('ENOTFOUND',    { host: 'no-such-host.invalid', port: 80, path: '/' })
await probe('EPIPE',        { host: '127.0.0.1', port: 55595, path: '/', method: 'POST',
                              headers: { 'transfer-encoding': 'chunked' } }, true)

// ETIMEDOUT: a listener whose accept queue is full silently drops further SYNs.
const stalled = `const net=require('net');const s=net.createServer(()=>{});
  s.listen({port:55593,host:'127.0.0.1',backlog:1},()=>{console.log('ready');
  const end=Date.now()+30000;while(Date.now()<end){}})`
const child = spawn(process.execPath, ['-e', stalled], { stdio: ['ignore', 'pipe', 'ignore'] })
await new Promise(r => child.stdout.on('data', d => String(d).includes('ready') && r()))
const primed = net.connect(55593, '127.0.0.1')       // takes the one accept slot
primed.on('error', () => {})
await new Promise(r => primed.on('connect', r))
await new Promise(done => {
  const t0 = Date.now()
  const s = net.connect(55593, '127.0.0.1')
  s.on('connect', () => done(log('  ETIMEDOUT     connected — backlog not full, rerun')))
  s.on('error', e => done(show('ETIMEDOUT', e, `  after ${Date.now() - t0} ms`)))
})
primed.destroy(); child.kill('SIGKILL')

// --------------------------------------------------- 4. timeout on a POST
log('\n=== 4. the same timeout, on a POST ===')
async function pay(key) {
  charges = []; seenKeys.clear()
  for (let attempt = 1; attempt <= 2; attempt++) {
    try {
      const r = await fetch('http://127.0.0.1:55591/charge', {
        method: 'POST', signal: AbortSignal.timeout(300),
        headers: key ? { 'idempotency-key': key } : {}
      })
      log(`  attempt ${attempt}: HTTP ${r.status} ${await r.text()}`); break
    } catch (e) { log(`  attempt ${attempt}: ${e.name} — no answer, retrying`) }
  }
  log(`  charges applied: ${charges.length} ${JSON.stringify(charges)}\n`)
}
log('  bare POST:');                      await pay(null)
log('  POST with Idempotency-Key:');      await pay('ord_5150')

api.close(); resetter.close(); closer.close(); process.exit(0)

node retry-lab.mjs

The whole run, on Node 23.5.0 / macOS 26.4:

=== 1. status codes ===
  code  retry  bare POST  Retry-After
  200   no     no         -
  400   no     no         -
  401   no     no         -
  403   no     no         -
  404   no     no         -
  408   yes    yes        -
  422   no     no         -
  425   yes    yes        -
  429   yes    yes        1
  500   yes    no         -
  502   yes    no         -
  503   yes    yes        1
  504   yes    no         -

=== 2. Retry-After vs your own backoff ===
  own backoff (50ms): 20 requests, 1033 ms
  obey Retry-After  : 2 requests, 1004 ms

=== 3. network errors, as node:http reports them ===
  label         code          syscall      GET   POST  message
  ECONNRESET    ECONNRESET    (none)       yes   no    "socket hang up"
  ECONNREFUSED  ECONNREFUSED  connect      yes   yes   "connect ECONNREFUSED 127.0.0.1:55594"
  ENOTFOUND     ENOTFOUND     getaddrinfo  no    no    "getaddrinfo ENOTFOUND no-such-host.invalid"
  EPIPE         EPIPE         write        yes   no    "write EPIPE"
  ETIMEDOUT     ETIMEDOUT     connect      yes   yes   "connect ETIMEDOUT 127.0.0.1:55593"  after 7818 ms

=== 4. the same timeout, on a POST ===
  bare POST:
  attempt 1: TimeoutError — no answer, retrying
  attempt 2: TimeoutError — no answer, retrying
  charges applied: 2 ["ch_1","ch_2"]

  POST with Idempotency-Key:
  attempt 1: TimeoutError — no answer, retrying
  attempt 2: HTTP 200 replayed ch_1
  charges applied: 1 ["ch_1"]

The two lines to watch are at the end. charges applied: 2 is what a bare POST with a retry loop does to a payment API on a slow afternoon.

Where this goes next

Classification is the first half of a retry policy. The second half — backoff, jitter, when to stop, and where a message goes when it can never succeed — is in retries, and why exactly-once delivery is a lie. Both are wired into every HTTP step in Workflow Builder, which is why a step that POSTs carries an idempotency key whether or not you asked for one.