# Metrics recipes: hosts, queues, business, Supabase

> Collector configs for Linux and Windows host metrics, queue depth and age gauges, business metrics, and Postgres metrics from the Supabase Metrics API.

Copy-ready setups for the metrics most apps need. All of them send OTLP metrics to the same endpoint with an ingest key that has **Metrics** ticked ([turn metrics on](https://belvador.dev/en/docs/otel/metrics#turn-them-on)).

The Collector recipes use standard `otelcol-contrib` components. Belvador's certification run covers the Collector forwarding OTLP metrics (all four types); the receivers below (`hostmetrics`, `prometheus`) are standard contrib components that run is not built around, so check the first points on the **Metrics** page after you start them.

## Host metrics on Linux

CPU, memory, load, disk, filesystem and network of the machine, one point a minute, from the Collector's `hostmetrics` receiver. Run one Collector per host.

```yaml title="host-metrics.yaml"
receivers:
  hostmetrics:
    collection_interval: 60s
    scrapers:
      cpu: {}
      memory: {}
      load: {}
      disk: {}
      filesystem: {}
      network: {}

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20
  resourcedetection:
    detectors: [env, system]
    system:
      hostname_sources: [os]
  resource:
    attributes:
      - key: service.name
        value: ${env:BELVADOR_HOST_SERVICE}
        action: upsert
      - key: deployment.environment.name
        value: ${env:BELVADOR_ENVIRONMENT}
        action: upsert
  batch: {}

exporters:
  otlphttp/belvador:
    endpoint: ${env:BELVADOR_OTLP_ENDPOINT}
    headers:
      Authorization: Bearer ${env:BELVADOR_INGEST_KEY}
    compression: gzip
    retry_on_failure:
      enabled: true
    sending_queue:
      enabled: true

service:
  pipelines:
    metrics:
      receivers: [hostmetrics]
      processors: [memory_limiter, resourcedetection, resource, batch]
      exporters: [otlphttp/belvador]

```

```bash
export BELVADOR_OTLP_ENDPOINT=https://ingest.belvador.dev/otlp/<project-id>
export BELVADOR_INGEST_KEY=<your-ingest-key>
export BELVADOR_HOST_SERVICE=web-1
export BELVADOR_ENVIRONMENT=production
otelcol-contrib --config host-metrics.yaml
```

- **`resourcedetection`** adds `host.name` from the operating system, so each machine is its own series.
- **`resource`** sets `service.name` (`BELVADOR_HOST_SERVICE`) and the environment, so host metrics filter like your app's data.
- In a container, the receiver sees the container, not the host. Mount the host's root read-only and set `root_path` on the receiver ([contrib docs](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver)).
- Already running the [forwarding Collector](https://belvador.dev/en/docs/otel/collector)? Add the `hostmetrics` receiver and the `metrics` pipeline to that file instead of running a second Collector.

A host sends a few dozen series; the default plan allows 10,000 per node.

## Host metrics on Windows

The same receiver runs on Windows. Windows has no `load` average; `paging` is added instead:

```yaml title="host-metrics.yaml"
receivers:
  hostmetrics:
    collection_interval: 60s
    scrapers:
      cpu: {}
      memory: {}
      paging: {}
      disk: {}
      filesystem: {}
      network: {}

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20
  resourcedetection:
    detectors: [env, system]
    system:
      hostname_sources: [os]
  resource:
    attributes:
      - key: service.name
        value: ${env:BELVADOR_HOST_SERVICE}
        action: upsert
      - key: deployment.environment.name
        value: ${env:BELVADOR_ENVIRONMENT}
        action: upsert
  batch: {}

exporters:
  otlphttp/belvador:
    endpoint: ${env:BELVADOR_OTLP_ENDPOINT}
    headers:
      Authorization: Bearer ${env:BELVADOR_INGEST_KEY}
    compression: gzip
    retry_on_failure:
      enabled: true
    sending_queue:
      enabled: true

service:
  pipelines:
    metrics:
      receivers: [hostmetrics]
      processors: [memory_limiter, resourcedetection, resource, batch]
      exporters: [otlphttp/belvador]

```

Install the Collector as a Windows service with its variables scoped to that service:

```powershell
# PowerShell as Administrator; otelcol-contrib.exe and host-metrics.yaml are in C:\otelcol
New-Service -Name otelcol -StartupType Automatic -BinaryPathName '"C:\otelcol\otelcol-contrib.exe" --config "C:\otelcol\host-metrics.yaml"'
# The variables belong to this service only (not the whole machine) and apply without a reboot.
Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\otelcol -Name Environment -Type MultiString -Value @(
  "BELVADOR_OTLP_ENDPOINT=https://ingest.belvador.dev/otlp/<project-id>",
  "BELVADOR_INGEST_KEY=<your-ingest-key>",
  "BELVADOR_HOST_SERVICE=win-1",
  "BELVADOR_ENVIRONMENT=production"
)
Start-Service otelcol
```

`otelcol-contrib.exe` comes from the [Collector releases](https://github.com/open-telemetry/opentelemetry-collector-releases/releases) (`otelcol-contrib_<version>_windows_amd64.tar.gz`). Belvador has not run this recipe on Windows itself.

## Queue depth and age

Two gauges tell you whether background work keeps up: how many jobs wait, and how long the oldest has waited. A growing age is the early sign of a stuck worker, before any job fails.

```js title="metrics.js"
import { metrics } from "@opentelemetry/api";

const meter = metrics.getMeter("app");
const attrs = { "messaging.system": "postgres", "messaging.destination.name": "emails" };

// How many jobs wait, and how long the oldest one has waited.
meter
  .createObservableGauge("queue.depth", { unit: "{message}", description: "Jobs waiting" })
  .addCallback(async (r) => r.observe(await queue.count("emails"), attrs));
meter
  .createObservableGauge("queue.oldest_age", { unit: "s", description: "Age of the oldest waiting job" })
  .addCallback(async (r) => r.observe(await queue.oldestAgeSeconds("emails"), attrs));

// A business signal: free connections in your own pool.
meter
  .createObservableGauge("app.pool.available", { unit: "{connection}" })
  .addCallback((r) => r.observe(pool.idleCount, { "db.client.connection.pool.name": "main" }));

// A counter you increment yourself; Belvador shows it as a rate.
export const signups = meter.createCounter("app.signups", { unit: "{signup}" });
// signups.add(1, { plan: "pro" });
```

- Observable gauges are read once per export (every 60 s), so the callback is the only code that runs. Keep it to one cheap query.
- Attributes follow the [messaging conventions](https://opentelemetry.io/docs/specs/semconv/messaging/) (`messaging.system`, `messaging.destination.name`): one series per queue.
- The code uses the global meter, so it works with the zero-code setup from the [Node.js recipe](https://belvador.dev/en/docs/otel/node) once `OTEL_METRICS_EXPORTER=otlp` is set. Python (`metrics.get_meter`), Java and Go have the same instruments.

Then alert: a **Metric** monitor on `queue.oldest_age`, aggregation `max`, *above* 600 over 5 minutes, grouped by `messaging.destination.name` — one alert per stuck queue ([Signal monitors](https://belvador.dev/en/docs/monitors/signals#metric-monitors)).

## Business metrics

Anything your code can count or read: sign-ups, payments, free connections in a pool, tokens a model used. Use a **Counter** for things that happen (`app.signups`, shown as a rate) and an **observable gauge** for levels (`app.pool.available`). Give them a short name with your own prefix, a unit, and attributes with few values (`plan`, not `user_id`).

## Supabase metrics

Supabase publishes the health and performance metrics of each project's Postgres database in Prometheus format (the [Metrics API](https://supabase.com/docs/guides/telemetry/metrics)). The Collector's `prometheus` receiver reads it once a minute and forwards it as OTLP:

```yaml title="supabase-metrics.yaml"
receivers:
  prometheus/supabase:
    config:
      scrape_configs:
        - job_name: supabase
          scrape_interval: 60s
          scrape_timeout: 30s
          scheme: https
          metrics_path: /customer/v1/privileged/metrics
          basic_auth:
            username: service_role
            password: ${env:SUPABASE_SECRET_KEY}
          static_configs:
            - targets: ["${env:SUPABASE_PROJECT_REF}.supabase.co"]

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20
  resource:
    attributes:
      - key: service.name
        value: supabase
        action: upsert
      - key: deployment.environment.name
        value: ${env:BELVADOR_ENVIRONMENT}
        action: upsert
  batch: {}

exporters:
  otlphttp/belvador:
    endpoint: ${env:BELVADOR_OTLP_ENDPOINT}
    headers:
      Authorization: Bearer ${env:BELVADOR_INGEST_KEY}
    compression: gzip
    retry_on_failure:
      enabled: true
    sending_queue:
      enabled: true

service:
  pipelines:
    metrics:
      receivers: [prometheus/supabase]
      processors: [memory_limiter, resource, batch]
      exporters: [otlphttp/belvador]

```

```bash
export BELVADOR_OTLP_ENDPOINT=https://ingest.belvador.dev/otlp/<project-id>
export BELVADOR_INGEST_KEY=<your-ingest-key>
export SUPABASE_PROJECT_REF=<project-ref>
export SUPABASE_SECRET_KEY=<sb_secret_… or service_role key>
export BELVADOR_ENVIRONMENT=production
otelcol-contrib --config supabase-metrics.yaml
```

- The endpoint uses HTTP Basic auth with the user `service_role` and a **secret** key. That key can read and write your whole database, so it stays on the Collector host — never in an app, a repository or Belvador.
- Run the Collector somewhere that is always on (a small VPS or a container next to your app); Supabase does not push these metrics.
- `service.name` is `supabase`, so these series sit apart from your app's.

## Other runtimes

Every OpenTelemetry SDK that exports OTLP/HTTP metrics reads the same `OTEL_METRICS_EXPORTER=otlp` variable: the Java agent and Python's `opentelemetry-instrument` export runtime metrics as soon as it is set; Go needs a `MeterProvider` with the `otlpmetrichttp` exporter in code. These exporters follow the same OTLP format the Collector run checks, but Belvador has not run them for metrics.
