10.3 Logging & observability (the log service, sinks, severities)

RS2 emits OTel-compatible structured logs at the host's single dispatch choke point, so logging is uniform and complete by construction. Where logs go is operator config; reading them back is a per-tenant log mount.

What gets logged (no config needed)

The host produces three kinds of records, all stamped with tenant/mount/service/trace:

• Boundary logs — one per dispatch, for external requests and internal pipeline hops, each with its own spanId under a shared traceId. Severity follows outcome: 5xx → Error (always emitted), 4xx → Warn, external success → Info, internal hop success → Debug. Attributes follow OTel semantic conventions (http.request.method, url.path, http.response.status_code, duration_ms, rs2.source, enduser.id, and on failure error.type/error.message/rs2.retryable).
• Service logs — a prebuilt service's own lines (e.g. auth logs failed logins), rs2.source: "service".
• Sandbox logs — a custom service's console.log/ctx.log (and Wasm log()), rs2.source: "custom", stamped with the invocation's trace so they line up with that request's boundary log (8.8).

A record is OTLP-shaped, one NDJSON line at rest:

{ "timeUnixNano": "…", "severityNumber": 13, "severityText": "WARN",
  "body": "GET /files/nope -> 404", "traceId": "…", "spanId": "…",
  "attributes": { "rs2.tenant": "acme", "http.response.status_code": 404,
                  "error.type": "not_found", "rs2.source": "external" } }

Where logs go — the logging block

This is operator config (serverConfig.json), not tenant config — physical log destination is node infra:

"logging": { "sink": "file", "level": "info",
             "file": { "path": "./logs", "maxBytes": 8388608, "backups": 5 } }

• sink: "file" writes per-tenant NDJSON (<path>/<tenant>.ndjson, size-rotated with backups); "none" disables logging.
• level: the boundary-log floor (debug/info/warn/error) — but 5xx always emit regardless of level.

The default file sink writes off the request path (a bounded channel + background writer), and a none sink is allocation-free on the hot path — so logging doesn't cost you latency.

Reading logs back — the log mount

Expose logs to a tenant with a log mount, scoped to that tenant (a tenant only ever sees its own):

{ "path": "/logs", "service": "log", "config": { "access": { "read": "A" } } }

Query params (all optional): $take (default 100, max 10000), severity (floor), traceId, service (a mount name, or a request-path prefix for boundary logs), since/until (Unix-ms or RFC 3339), q (case-sensitive body substring).

Guard the mount like any operational surface (e.g. access: { "read": "A" }).

Exporting to an observability platform

Swapping the sink for a write-only OTLP/observability exporter is a documented follow-on: ship logs to your platform, and the log reader degrades gracefully to "exported, not locally queryable" (501). The X-Trace-Id response header correlates a response to its line either way.

Next: editing tenant config safely on a running node.

← Previous: 10.2 Single- vs. multi-tenant · Manual home · Next: 10.4 Editing config safely →