Decision record · ADR
ADR-0006: Defense bar for operator-authored inputs and review-finding triage
Status: accepted | Date: 2026-07-04
References: RFC-0004:C-WORKLOAD-PROFILING, RFC-0002:C-WORKSPACE-AUTHORITY
Context
Section titled “Context”InferLab is an experiment tool: workspace TOML, serve profiles, and profiler escape options are authored by the operator who runs the tool, not by an adversary. External design reviews repeatedly probe the load gates that validate these inputs, and every probe of a blocklist-style gate finds a bypass, because a blocklist over an external tool’s option surface can never be complete.
Successive reviews of the profiler escape gate demonstrated the failure mode: each confirmed bypass was converted into a new rejection layer (attached short forms, abbreviated long forms, positional terminators), the layers acquired their own defensive test apparatus (a hand-copied option-surface snapshot claiming upgrade protection, a chmod-based failure-injection test), and the apparatus itself was then falsified by the next review — the surface snapshot was provably incomplete against the very binary it described, the chmod test inverts under uid 0, and a proposed replacement parser for nsys help text was falsified by a bracket-combined help token. Confirming that a gap was constructible was treated as sufficient reason for more defense, without asking whether the gap occurs in a real operator workflow or what its failure mode costs. The project’s design principles already hold that machine-local conveniences need no integrity defenses against hypothetical tampering; this decision extends the same posture to operator-authored inputs and to review triage.
Decision
Section titled “Decision”We will admit a new rejection layer over operator-authored inputs only when the mistake they catch would SILENTLY corrupt recorded evidence — the capture spine (session identity, report lifecycle, range mechanics) or the source identity (digest coverage of effective bytes). Two categories never justify a new layer: mistakes the underlying tool itself rejects loudly at runtime, and collisions the managed values win by documented splice semantics. Findings in those categories are recorded as candidate-list entries at most.
External review findings receive a three-value triage, never two: confirmed and worth fixing; confirmed but not worth fixing, with the positioning rationale recorded where the finding was reported; or refuted by evidence. Confirming that a shape is constructible does not by itself make it work worth doing — the finding must matter in a real operator workflow, judged by whether real tools (editable installs, package managers, framework build systems) produce the shape or only deliberate construction does.
No qualification apparatus is built to defend a gate against hypothetical future versions of an external tool. Version qualification is a manual, documented step: the gate’s documentation records what was checked against which version, and the check is repeated by hand when the qualified version changes. A wrong gate verdict after a version change is an acceptable failure mode when it is a loud load error with a clear message and a small fix; it is not worth a standing parser over another tool’s help text.
The source-identity invariant itself — the digest must cover the bytes frameworks actually read — remains fully defended: silent irreproducibility is the worst failure an experiment tool can have. This decision governs marginal shapes and gate extensions, not the invariant.
Consequences
Section titled “Consequences”Positive
Section titled “Positive”- Reviews stop ratcheting: a finding against a gate is declined with a recorded rationale when it fails the silent-corruption bar, instead of growing the blocklist and its test apparatus.
- Gate apparatus that fails the bar is removed rather than maintained; shipped, spec-backed rejection layers stay because they are verified free of false positives and removing shipped behavior is more churn than keeping it.
- Version qualification of external tools stays a deliberate human step with its scope recorded on the gate’s documentation, instead of a standing parser over another tool’s help output.
Negative
Section titled “Negative”- An operator can still write escape mistakes the gate does not catch; the mitigation is that the underlying tool rejects those loudly at runtime, which is exactly the class the bar declines to pre-empt.
- A stale gate verdict after an nsys version change surfaces as a loud, well-labelled load error until the manual re-check runs; the accepted cost is a one-line table fix.
- The silent-corruption bar itself can be misjudged: a finding declined as confirmed-but-not-worth-fixing may later prove to be a real silent-corruption class. The mitigation is the re-entry condition — a declined shape observed in a real operator workflow, or shown to corrupt recorded evidence silently, re-enters triage immediately; declined findings remain on record with their rationales in the project’s internal review notes.
Neutral
Section titled “Neutral”- Three-value triage verdicts and their rationales are recorded in the project’s internal review notes at the point where each finding was triaged; the operator adjudicates a disputed verdict.
- The posture extends the project design principle that machine-local conveniences need no integrity defenses against hypothetical tampering; future reviews cite this ADR when declining defense-of-defense work.
Alternatives Considered
Section titled “Alternatives Considered”Gate new rejection layers on silent evidence corruption, triage external findings three ways, and replace gate-protecting qualification apparatus with a manual documented re-check (accepted)
Section titled “Gate new rejection layers on silent evidence corruption, triage external findings three ways, and replace gate-protecting qualification apparatus with a manual documented re-check (accepted)”- Pros: Stops the blocklist ratchet at a principled line: the gate serves the operator’s evidence, not its own completeness, Review effort concentrates on findings that occur in real workflows, with declines recorded and citable, No standing parser over another tool’s help text to maintain; version qualification stays a deliberate human step
- Cons: An operator can still write escape mistakes the gate does not catch, and learns from the tool’s own runtime error instead, A stale gate verdict after an nsys version change surfaces as a load error until the manual re-check runs
Keep hardening the gate and its tests after every review round, including automated qualification apparatus over the external tool’s help output (rejected)
Section titled “Keep hardening the gate and its tests after every review round, including automated qualification apparatus over the external tool’s help output (rejected)”- Pros: Each individual layer is empirically justified by a confirmed bypass, Maximal load-time coverage of operator mistakes
- Cons: A blocklist over an external option surface is never complete, so the ratchet has no terminal state, The apparatus itself becomes the defect surface: the hand-copied table was provably incomplete and the replacement parser was falsified within one review, Cost lands on every future review; the protected failure mode was already a loud load error plus a one-line fix
Roll back the shipped abbreviation, attached-form, and terminator rejections entirely and keep only exact-name matching (rejected)
Section titled “Roll back the shipped abbreviation, attached-form, and terminator rejections entirely and keep only exact-name matching (rejected)”- Pros: Smallest possible gate; nsys itself rejects most nonsense loudly
- Cons: Removes shipped, spec-backed behavior that is verified free of false positives — more churn than keeping it, The terminator class is genuinely silent corruption (nsys start – exits 0, drops the managed tail, leaves a stray agent) and must stay