Skip to content

Monitors: know when background work fails or never runs

Monitors watch scheduled jobs, queues and workers: a run that fails, runs too long, ends with an unknown result or never happens opens an incident.

Markdown
On this page

Errors tell you when code throws. They cannot tell you that a nightly job never started, that a queue worker died, or that an import ended without anyone knowing whether it finished. A monitor watches one piece of background work and opens an incident when a run goes wrong — including the run that never happens.

Prefer no SDK at all? A ping URL monitor watches any cron job or timer with one secret URL, and keeps alerting even when your app or node is down.

Check-in monitorPing URLUptime check
Who sendsYour job, through the SDK or HTTPYour job, with one HTTP callBelvador's probes call your URL
CatchesMissed, failed and too-long runsMissed, failed and too-long runsA site or port that is down, slow, wrong or has an expiring certificate
Judged byYour nodeBelvador's monitor serviceBelvador's monitor service, from three regions
While your node is downShows Not observable; a real miss is reported lateStill alertsStill alerts

How a monitor works#

Your job sends a short check-in to Belvador when it starts and when it ends. Your node compares those check-ins with what the monitor expects and decides, every 15 seconds, whether each run was healthy.

  • A check-in is not an error event and never creates an issue. Errors your job throws still become issues as usual, linked to the run they happened in.

  • Monitors live per project. The slug (for example nightly-invoices) is how your code names the monitor; it cannot change later.

  • Check-ins come from servers, workers, CI or a shell. Browsers cannot send them.

Three kinds of schedule#

KindUse it forWhen a run is late
Cron scheduleJobs on a calendar: 0 2 * * * = every day at 02:00, in the time zone you chooseNo check-in by the scheduled time plus the grace period
Every N minutes/hoursWorkers and heartbeats that should report regularlyNo check-in within N after the last one, plus the grace period
When something happensWork started by an event (a webhook, a button)Never late — only failures and timeouts count

Cron schedules follow the time zone's clock changes: a 02:30 job on the night the clocks skip 02:30 is due an hour later, at 03:30, and a repeated hour runs once. The create form shows the next five deadlines so you can check this.

What a run can end as#

ResultMeaningCounts as
OKThe job finished its workHealthy
SkippedThe job ran but had nothing to do (your code said so)Healthy
FailedThe job reported an error or threwFailing
Timed outIt started but did not finish within the maximum runtimeFailing
MissedNo check-in arrived in timeFailing
Outcome unknownThe job ended but could not tell whether its work took effect — for example a payment provider timed out after the request was sentNeeds review

A retried attempt belongs to the same run: three attempts where the last one succeeds is one healthy run with three attempts, not two failures.

Limits per monitor#

SettingDefaultRange
Grace period — how late a check-in may arrive5 min1–1440 min
Max runtime — after this a started run is timed out30 min1–10080 min (7 days)
Failures before an incident11–720
Successes before recovery11–720
Environments to watchproductionup to 10

Environments#

A monitor watches the environments you list (default production). Check-ins from other environments — staging, development — are still recorded and shown in their own tab, but a missed run there never opens an incident. A laptop that is switched off at night therefore never pages anyone.

Incidents and alerts#

One incident opens when a monitor starts failing and closes when it recovers. The default alert rule Monitor problems sends exactly one message when it opens and one when it closes, to every active destination of the workspace — see Alerts. You can mute a single monitor: incidents still open on the page, but no message is sent.

Needs review is amber, not red: someone has to check whether the work happened. Open the run, check your system, then press Mark as reviewed.

When Belvador cannot see#

If your node is offline, is catching up after a reconnect, or check-ins were rate limited, the monitor shows Not observable instead of Missed. Belvador never marks a run as missed for a time it could not watch. After the node is back it waits up to 10 minutes for check-ins buffered at the edge before judging, so runs that happened while it was away appear at the right time; a run that really did not happen is then reported with "detected late".

Your app is never slowed down by this: check-ins are sent in the background with a small queue, and a Belvador outage only means check-ins are dropped and counted.

Next steps#

Spotted something wrong or missing? Tell us: hello@belvador.dev