Skip to content

Engineering

Retried jobs, duplicate errors: what each attempt sends to your tracker

We retried a failing payment job with the Sentry Node SDK and counted every event. Ten failed jobs sent 30 events; one that recovered still sent 2. Dedupe did not help.

Andrew Babo6 min read

Share
On this page
Retried jobs, duplicate errors: what each attempt sends to your tracker

Every attempt you capture is an event. In our lab, ten payment jobs that each failed three times sent 30 events for 10 failures, and a job that recovered on its third try still sent 2 events for something that worked. The tracker grouped all of them into one issue, so the list looked fine.

The counts did not: escalation thresholds, "archive until 100 more events" and your event quota all count attempts, not jobs. Report the final failure, put the attempt count on it, and let the job monitor record the attempts.

The lab#

On 26 October 2026 we wrote a job that calls a fake payment provider which times out, and a runner that tries it three times with a short backoff. A fresh Error is thrown on every attempt, as a real timeout would be. We ran nine reporting patterns with the official Node SDK (@sentry/node 10.76.2) at its defaults, and again with its Dedupe integration added. Each pattern ran in a fresh Node.js 22 process, so no state carried over, and a local collector kept every event the SDK sent.

To count issues, we passed each event through the same normalization and grouping code that Belvador's node runs on arrival. Three runs each; every run gave the same numbers.

What each retry pattern sent
Lab 26 Oct 2026 · Node 22.23.4 · @sentry/node 10.76.2 · 3 runs each, identical
PatternEvents, SDK defaultsEvents, + DedupeIssues
One job fails 3 times, capture every attempt311
Same, with the attempt as a tag311
Same, wrapped as attempt N of 3 failed311, a different one
One job fails 3 times, capture the final failure111
Final failure wrapped as charge failed after 3 attempts111, a different one
Job recovers on attempt 3, capture every attempt211
Job recovers on attempt 3, capture the final failure000
Ten jobs fail at once, capture every attempt30301
Ten jobs fail at once, capture the final failure10101

Grouping did its job#

Every attempt of the same failure landed in one issue, including ten jobs with ten different order numbers in the message. The grouping key is the stack trace, and the message is not part of it, so timeout for order A-1001 and timeout for order A-1010 are one problem. If you were worried that retries would flood your issue list with copies, they did not.

The counts did not#

An issue's event count is what the rest of the system reads:

  • Escalation. Belvador marks an issue escalating when its events in the last hour reach 10, or five times its usual hourly rate if that is higher. With three captured attempts per job, 4 failed jobs reach that floor; with one event per job it takes 10.
  • Archive conditions. "Archive until 100 more events" comes back after 34 failed jobs instead of 100.
  • Recovered work. The job that succeeded on its third try left two error events behind. On a busy queue with a flaky provider, that is a steady stream of events for work that finished.
  • Quota. Events are what you pay for or what fills your disk, depending on the product.
Events sent, by reporting pattern
Lab 26 Oct 2026 · @sentry/node 10.76.2 · events counted at the collector

Dedupe is not a retry filter#

In our browser lab, Sentry's Dedupe integration turned 20 identical throws into one event, so it is tempting to reach for it here. Two things got in the way.

First, the Node SDK does not turn it on. @sentry/node 10.76.2's default integrations do not include Dedupe; the browser SDK's do. On a server, every attempt goes out unless you add it.

Second, added by hand, it only helped the single job. Dedupe compares each event with the one just before it, as the Sentry docs describe. One job's three attempts in a row became one event. Ten jobs failing at the same time still sent 30: their attempts interleave, and each carries a different order number, so no event matches the previous one. It also dropped our attempt 2 of 3 failed events even though their messages differ, because it compares the first error in the chain, which was the identical cause underneath. That is a quota guard for tight loops, not a rule about which attempts matter.

Report the final failure#

The runner knows which attempt is the last one, so it is the right place to decide:

TypeScript
async function runWithRetries(job, input, { attempts = 3 } = {}) {  let last;  for (let attempt = 1; attempt <= attempts; attempt++) {    try {      return await job(input);    } catch (err) {      last = err;      if (attempt < attempts) await sleep(backoff(attempt));    }  }  Sentry.captureException(last, { tags: { attempts: String(attempts) } });  throw last;}

In the lab this sent one event per failed job and none for the job that recovered, and those events joined the same issue as the per-attempt ones, so switching patterns does not split your history.

Wrapping the final error is a choice, not a default. new Error("charge failed after 3 attempts", { cause: last }) keeps the cause in the event, but it starts a new issue, because its stack begins in the runner instead of in the payment call. Pick one shape and keep it: the same failure reported sometimes wrapped and sometimes not becomes two issues.

Other ecosystems settled on the same answer. Sentry's Ruby SDK reports an Active Job failure after its retry_on attempts are exhausted by default, and offers report_after_job_retries for Sidekiq and Delayed Job, as listed in its integration options and introduced in this pull request.

Checklist#

  1. Capture in the runner, after the last attempt, not inside the job.
  2. Put the attempt count on the event as a tag, so the issue shows how hard it tried.
  3. Keep one shape: wrapped or not, but the same every time.
  4. Do not count on Dedupe: it is off on Node and it cannot tell attempts from jobs.
  5. Record attempts somewhere that is built for them: a job monitor, which knows that three attempts are one run.
  6. If you do capture every attempt, read issue counts as attempts, not as failed jobs.

Where Belvador fits#

Belvador's withMonitor records the attempts of a run together when you pass runId, attempt and willRetry: an attempt that will be retried shows Will retry and opens no incident. Until today it also reported the error of every failed attempt, which is the "every attempt" row above. We found that in this lab and shipped SDK 0.5.3 the same day: reportRetries: false sends only the final failure as an issue event, and the error carries the attempt number. The default stays as it was, so the run still shows what each attempt hit if you want that.

How issues are grouped, and what escalating means, is in the issues docs. The background work guide covers where to report for queues, webhooks and long-running workers, and the cron monitoring page shows the monitors side by side.

Keep reading#

Part of the guide: Monitoring AI agents that run on serverless functions

All posts

Guides

Monitoring AI agents that run on serverless functions

An agent run on serverless is a chain of short invocations, each with its own time limit. Here is what to watch at every link, with three small experiments to show where the signal gets lost.

Andrew Babo6 min read

Read next in the docs

See your own production errors this way

Belvador runs on your own Hetzner server: errors, cron jobs, uptime and traces, with your data staying yours.