Observability

OpenTelemetry, and only OpenTelemetry. Traces, metrics and logs all leave over OTLP, so whatever you already run is what you use — the runtime has no opinion beyond the protocol.

There is no /metrics endpoint. Metrics are pushed over OTLP like everything else. One telemetry protocol in the binary, and a Collector is what turns it into whatever you actually scrape. Set RUNTIME_OTLP_ENDPOINT and point it at one; leave it unset and everything goes to stdout, which is what you want on a laptop and never in production.

An unreachable collector is not a startup error. A Grafana outage must never become an application outage.

Three log streams, one stdout

Told apart by target, not destination:

Without that split, a customer’s log("retrying") pages your on-call engineer. Route them differently.

The audit stream

{
  "operation": "delete",
  "entity": "sales.OrderLine",
  "row": "01a0…",
  "subject": "sub-clerk",
  "constrained": false,
  "cascaded": true
}

cascaded tells you the row was reached by a cascading delete rather than asked for, so “what did that delete actually take with it” is answerable afterwards. constrained says whether the caller’s row constraint was in play — which is how you find an entity that is accidentally unconstrained.

Records are emitted after the transaction commits. A rolled-back write never appears, because an audit log that reports things which did not happen is wrong in the direction that makes people stop trusting it.

Traces

Spans carry the route template — /api/v1/{module}/{entity}/{id} — never the concrete URL. One time series per route rather than one per entity, which is the difference between working instrumentation and taking your own backend down with it. Unmatched requests are bucketed under a single unmatched label, so a scanner probing random paths cannot mint series by guessing.

Incoming traceparent headers are honoured, so a request continues the caller’s trace instead of starting a new one.

Business metrics

Applications declare their own counters in metrics.yaml, and the runtime maintains them:

sales.orders.written_total{status="Placed"}     2
sales.orders.cancelled_total                    1

These arrive alongside the platform’s own metrics with the names the application author chose. Labels are restricted to enum and boolean attributes — the only types whose value set the model declares — so a model that would explode your backend fails validation rather than producing a bill.

That guard is enforced at model load, not at runtime. You do not have to trust application authors to get cardinality right; the model does not load if they did not.

Correlating the three

The local stack in the deployment repository wires this up as an example: Prometheus for metrics, Tempo for traces, Loki for logs, Grafana over all three, with trace-to-log and log-to-trace links already configured.

The pipeline is OTLP end to end — the runtime pushes to the Collector, the Collector pushes to all three, because Prometheus 3, Tempo and Loki 3 all ingest it natively. Swapping any of them for a vendor’s SaaS is a change to the Collector’s exporters and nothing else. That is the entire point of the runtime speaking only OTLP.