Skip to content

Decision record · ADR

ADR-0026: Extend pinned AIPerf through a release-owned session plugin

Status: rejected | Date: 2026-07-31

References: RFC-0004:C-BENCH-LINEAR-SESSIONS, RFC-0004:C-INFERLAB-TOOLCHAIN, RFC-0005:C-BENCH-LINEAR-SESSION-EVIDENCE, ADR-0003, ADR-0023

The release-pinned AIPerf 0.11 runtime already owns native multi-turn dataset loading, live assistant-response history, inter-turn delay scheduling, complete-session concurrency, phase separation, and transport metrics. The linear-session RFC additionally requires pre-transport content bounds, client chat-template rendering over live history, terminal session failure, unchanged warmup inputs, and evidence for suppressed turns. Inspection of the pinned runtime found that its ordinary configuration surface cannot enforce those additional semantics: a failed non-final request still dispatches the next turn, warmup mutates the system message, the completions endpoint rejects accumulated turns, and no pre-send context hook exists.

ADR-0003 selected a release-pinned CLI, generated configuration, and artifact boundary instead of importing arbitrary AIPerf library state into the Bench runner. The newer AIPerf release exposes a plugin registry for endpoints and services, but satisfying terminal failure requires a narrow extension at the worker lifecycle rather than another configuration field. The decision therefore needs to preserve AIPerf as the measurement and scheduling engine, keep the Rust control plane framework-neutral, and contain version-sensitive Python code without pretending that the unextended CLI satisfies the RFC.

We will extend session-backed Bench cases with one release-owned AIPerf plugin that is bound to the exact AIPerf version in the InferLab measurement toolchain. The existing Bench runner continues to generate the native configuration, invoke AIPerf’s CLI, and normalize unchanged native artifacts. The session-specific launcher registers the plugin before entering that CLI, while independent-request cases keep the ordinary executable path.

The plugin owns only the gaps that require live session state: pre-transport content measurement and optional prompt rendering, response validity, terminal failure and later-turn suppression, and removal of AIPerf’s warmup-only prompt mutation. It consumes the already frozen measurement request and population references rather than introducing a public plugin selector or another configuration authority. Version-sensitive imports remain isolated in the Bench integration, are included in the runner identity, and fail closed when the installed AIPerf version differs from the version the shim qualifies.

This direction wins because AIPerf remains the native phase, scheduler, transport, raw-record, and metric owner while the compatibility surface is explicit and release-scoped. It refines ADR-0003 for the demonstrated session workflow; it does not turn arbitrary AIPerf Python APIs into a general InferLab extension surface.

Positive: linear sessions reuse the same AIPerf load, transport, metrics, and artifact pipeline as independent Bench cases, and the Rust control plane remains unaware of AIPerf implementation nouns. The extension is activated only by the resolved session workflow and travels in the same content identity as the Bench runner.

Negative: the compatibility layer depends on selected endpoint and worker lifecycle hooks whose stability is bounded by the AIPerf pin. Each AIPerf upgrade therefore carries focused source review, offline boundary tests, and a real session qualification cost; a mismatched pin is handled by refusing session execution rather than guessing compatibility.

Neutral: AIPerf remains an unmodified third-party distribution, ordinary Bench behavior retains the direct CLI path, and framework integrations continue to own only server capabilities. The RFCs remain authoritative for session behavior and evidence; this decision records where the version-sensitive translation lives.

Load one release-owned, version-pinned session plugin through AIPerf’s plugin registry and continue invoking the native CLI (accepted)

Section titled “Load one release-owned, version-pinned session plugin through AIPerf’s plugin registry and continue invoking the native CLI (accepted)”
  • Pros: Retains AIPerf’s native phases, scheduler, transport, raw records, and metric pipeline, Localizes version-sensitive hooks inside the release-owned Bench integration and activates them only for session-backed cases, Keeps session semantics out of Rust and out of framework integrations
  • Cons: Couples the plugin to the exact AIPerf plugin and worker lifecycle shipped by the pinned release, Requires plugin discovery and cross-process behavior to be qualified again for every AIPerf pin

Implement the session scheduler and HTTP client directly in the InferLab Bench runner (rejected)

Section titled “Implement the session scheduler and HTTP client directly in the InferLab Bench runner (rejected)”
  • Pros: Gives InferLab direct control over history, bounds, delays, and failure handling
  • Cons: Creates a second transport and timing engine beside AIPerf, Makes request metrics and phase behavior diverge from ordinary InferLab Bench cases
  • Rejected because: It would undo the common-client boundary that motivated AIPerf adoption and would not be an AIPerf-native session phase.

Wait for a future AIPerf release to expose every required hook before enabling sessions (rejected)

Section titled “Wait for a future AIPerf release to expose every required hook before enabling sessions (rejected)”
  • Pros: Avoids maintaining a local compatibility layer against the current worker lifecycle
  • Cons: Leaves the governed operator workflow unavailable despite the current release exposing a bounded plugin mechanism, Conflates this feature with a broader AIPerf upgrade and requalification
  • Rejected because: The current pinned release provides enough native scheduling plus an explicit extension registry; waiting does not reduce the present product need.

Patch or vendor AIPerf’s installed source tree (rejected)

Section titled “Patch or vendor AIPerf’s installed source tree (rejected)”
  • Pros: Can change any internal behavior without designing an extension boundary
  • Cons: Turns the measurement toolchain into an unpublished fork and obscures upstream identity, Expands maintenance and supply-chain review far beyond the session workflow
  • Rejected because: A versioned plugin is a smaller and auditable compatibility surface than modifying the pinned distribution.