How production errors, cron jobs and traces really behave, with code you can run.
Engineering
When a tool fails, the agent hands the error to the model and the request still succeeds. Mark the tool span, record the exception once, and report it from the handler.
Andrew Babo3 min read
Engineering
We built one module with each of Vite's three source map settings and threw an error from each build. One gave a useless stack frame; the other two shipped our whole source code in a file.
Andrew Babo4 min read
Engineering
Every failing fix in our test said it was fixed, came with a passing test and rated itself 92 to 100. Only running the customers' input showed which still crashed. A protocol for your own bugs.
Andrew Babo5 min read
Engineering
We ran a support agent 60 times with seeded failures. Every failed refund returned HTTP 200 and reached no error tracker. Only the trace showed it, and customers heard the truth in 13 of 30 failures.
Andrew Babo4 min read
Engineering
We stopped a job with SIGTERM and SIGKILL, three times each. One common way of pinging reported success for a job that had been killed.
Andrew Babo6 min read
Engineering
We deployed a Worker that fails in five different ways and watched what the client, Cloudflare and an error tracker each saw. Two of the five returned 200, and one of those left no trace anywhere.
Andrew Babo5 min read
Engineering
We gave three AI models an error tracker over MCP and one question. All found the right issue. Only some found the cause, and with a write tool most resolved a bug that was still live.
Andrew Babo7 min read
Engineering
We ran 14 kinds of browser failure through the Sentry browser SDK and our own, three times each. A caught error that is only logged sent nothing; a string rejection arrived with no stack.
Andrew Babo5 min read
Engineering
We put a fake secret into 14 real Node.js errors and reported them with Sentry's Node SDK at its defaults. 13 of 15 secrets left the process, in the message, a breadcrumb or the source lines.
Andrew Babo6 min read
Engineering
We replayed four synthetic incidents through the SLO evaluator Belvador runs every minute. A plain threshold alert on the same traffic fired 16 times for one slow leak.
Andrew Babo6 min read
Engineering
A route loader that throws during server rendering reaches the browser as a message and nothing else. We traced one error through three setups and counted what an error tracker received each time.
Andrew Babo5 min read
Engineering
We ran Belvador's scrubber on 16 synthetic error samples, fixed the three gaps that were ours and ran it again. Secrets with a known shape went; phones and a name still got through.
Andrew Babo5 min read
