7.8 Segments and the ?$plan introspection

To retry safely, the runtime partitions a serial pipeline into segments. The segment is the atomic retry unit, and understanding it is the difference between a pipeline that recovers cleanly and one that double-charges a card. ?$plan shows you the segmentation before you run.

What a segment is

The executor splits a serial pipeline at materialization points — transforms and splits force a boundary, because they need the body in hand. Each run of steps between boundaries is a segment.

The segment is the atomic retry unit: a retryable failure mid-segment re-runs the whole segment from its materialized input.

Within a segment, keyed/unsafe calls get the auto-derived idempotency keys (7.7) that are stable across attempts, so re-running the segment doesn't duplicate their effects.

Why the boundary placement matters

Because a retry re-runs the entire segment, you want unsafe effects to be the last thing in their segment — otherwise a later failure re-executes them. Consider:

GET /fetch        ─┐
POST /charge       │  one segment — a retry re-runs BOTH,
GET  /confirm     ─┘  re-charging unless /charge is keyed

vs.

GET /fetch
{ transform }      ← boundary
POST /charge       ← its own segment; retry re-runs only this

Putting a transform before the POST gives the charge its own segment, so a failure after it doesn't re-run the fetch, and a failure of it re-runs only the charge (safely, with its derived key).

?$plan — see the segmentation

GET /<mount>/.pipelines/<spec>?$plan

returns { pipeline, plan: { segments: [{start, end, checkpointEligible}], warnings } }. Two things to read:

• segments — the retry boundaries the runtime computed.
• warnings — most importantly, "unsafe-effect step is not the last in its segment." That warning means a segment retry would re-execute that step. Fix it by marking the step "effect": "keyed", moving it to a segment end (e.g. insert a transform before it), or knowingly accepting the duplication risk. Also here: "external host '…' is not covered by any httpOut grant on this mount" — a literal absolute call URL (7.3) that would be denied at execution; add a matching httpOut grant to the mount. (Hosts built by ${…} interpolation can't be checked statically.)

Streaming bodies opt out of snapshotting

A segment whose input is a large or unknown-size streaming body (default threshold 1 MB) is not snapshotted — it runs at most once. Streaming is preserved at the cost of that segment's retryability. So a pipeline that streams a big upload through an effect won't retry that part; if you need retry there, ensure the body is materialized (small, or transformed) first.

?$plan is the tool that turns all of this from guesswork into something you can inspect. The last section of this part is a practical debugging workflow.

← Previous: 7.7 Retries, effects, idempotency · Manual home · Next: 7.9 Debugging a pipeline →