9.5 Limits, containment, and the per-tenant circuit breaker
RS2 is built to run untrusted and multi-tenant workloads, so resource limits aren't an afterthought — they're enforced by the host on every invocation, and a tenant that misbehaves is contained so it can't degrade its neighbors.
The default limits
Per-invocation / per-tenant (operator-configurable):
| Limit | Default |
|---|---|
| Wall clock — service | 30 s |
| Wall clock — pipeline | 120 s |
| Memory | 128 MB |
| Materialized body | 100 MB |
| Concurrent invocations per tenant | 64 (excess fails fast — no queueing) |
| Outbound calls per invocation | 64 |
| Call depth | 16 |
| Pipeline fan-out | 1000 |
A breach returns limit_exceeded (9.2) naming the limit in the limit field,
with observed/cap and a retryAfterMs. It's a structured 503, never a
crash.
Fail-fast admission, not queueing
When a tenant is at its concurrency cap, further requests fail fast with 503
rather than queueing. This is deliberate: queueing under overload turns latency
into a cascading failure; fast rejection keeps the system predictable and lets a
caller back off.
The per-tenant circuit breaker
Repeated resource breaches (default 8 within 10 s) trip a per-tenant circuit
breaker: subsequent requests fail fast with limit: "tenant_breaker" and a
Retry-After for a cooldown (default 5 s). This stops pathological code from
re-occupying engine threads over and over.
Importantly, admission rejections don't feed the breaker — only genuine wall-clock/memory/materialization breaches do. So a tenant that's merely busy isn't punished; one that's actually running runaway code is.
Why this gives real containment
These mechanisms stack: the isolate's wall-clock/heap kill stops a single bad
invocation; per-tenant concurrency admission bounds how many can run; the breaker
stops a flood of breaches from monopolizing threads. The net effect — measured
under hostile load (an infinite loop and an allocation bomb flooding a node) — is
that a neighbor tenant's latency stays close to baseline while the attacker's
requests all return structured 503s. A bad tenant degrades itself, not the
node.
What this means for you
- Treat
limit_exceededas a normal, retryable condition — respectRetry-After/retryAfterMs. - Design custom services to finish well within the wall-clock and memory bounds (8.8); remember virtual timers don't consume wall time, but busy loops do.
- If you're an operator, these defaults are tunable — see Part 10 and Appendix D.
Last in this part: how every request is traced and correlated to its logs.
← Previous: 9.4 Caching · Manual home · Next: 9.6 Tracing and log correlation →