Skip to content

Decision record · ADR

ADR-0035: Delegate SemiAnalysis AgentX trace scheduling to AIPerf

Status: accepted | Date: 2026-08-06

References: RFC-0004, RFC-0005, ADR-0003, ADR-0017, ADR-0024

SemiAnalysis InferenceX now treats AgentX as a high-priority serving benchmark. Its corpus is not a list of independent prompts or a finite linear conversation: each trace contains a root session, nested subagent request streams, recorded spawn and join dependencies, synthetic cache-block geometry, and a source timeline. Profiling begins from a sampled point inside each trajectory, first reconstructs the pre-boundary cache state, and then measures a duration-driven continuation. Declared concurrency counts root session-tree lanes even though descendants can create more simultaneous HTTP requests.

InferLab already owns the operator-facing Bench definition, release-qualified source catalogs, resolved experiment policy, case lifecycle, normalization, and durable records. Its existing linear-session path intentionally uses live server responses to build later turns and requires every planned turn to finish; those semantics do not describe AgentX source-response replay, branches, trajectory snapshot warmup, or duration truncation. Reusing the same source variant would make two different load units and response authorities appear equivalent.

AIPerf 0.12 contains a dedicated AgentX scenario, Weka trace materializer, session-tree registry, branch scheduler, trajectory snapshot and cache-pressure warmup, duration replay, raw transport records, branch aggregates, and native submission-validity checks. Reimplementing those rules in Rust would create a second scheduler and a second interpretation of upstream trace semantics. Calling the InferenceX shell script directly would instead surrender InferLab’s resolved configuration, records, toolchain pin, and engine-neutral Bench boundary to a moving upstream script.

The design question is therefore where the recorded session-tree scheduling authority belongs, and how much of its underlying mechanism should become public InferLab configuration.

Use a distinct, closed agentic_source Bench boundary for the release-qualified SemiAnalysis AgentX workflow. InferLab owns the public dataset and profile names, immutable source identity, resolved concurrency, duration, seed and endpoint policy, case validation, normalized interpretation, and durable evidence. The profile freezes the relevant InferenceX reproduction policy instead of projecting AIPerf scenario, loader, timing, warmup, cache-bust, failure-threshold, or DAG knobs into workspace configuration.

Delegate materialization and runtime parent/subagent scheduling to the release-pinned AIPerf 0.12 AgentX capability. Its native materialized input is the topology authority; its scheduler owns trajectory boundaries, snapshot and cache-pressure warmup, recorded timing, spawn and join behavior, lane recycling, duration finalization, and native failure and scenario outcomes. InferLab validates and reconciles those artifacts but does not independently reconstruct the graph or recompute acceptance thresholds.

Keep this boundary separate from linear sessions. Linear sessions continue to feed live responses into a fixed next turn and use finite complete-session counts. AgentX replay uses source-derived synthetic response context, recorded branches, and a duration window; it measures transport behavior rather than semantic agent-task success. This decision does not introduce a general InferLab workflow or DAG runtime.

Positive: operators get one reproducible AgentX Bench definition across supported serving engines while retaining InferLab-owned resolution, lifecycle, records, server metrics, and normalized transport metrics. The release profile can match a qualified InferenceX dataset and script revision without exposing unstable AIPerf implementation names. Native AIPerf artifacts remain the single authority for source topology, scheduling, branch outcomes, and raw requests, avoiding duplicate graph logic and evidence drift.

Negative: the initial public surface is intentionally specific to qualified AgentX trace profiles. Adding another agentic trace format or a materially different live agent workflow requires evidence that it fits this boundary or a new design decision. InferLab depends on the pinned AIPerf AgentX artifact contract and must qualify any AIPerf upgrade before changing its lowering or record parser. A moving InferenceX main branch can exceed the supported profile until a new release qualification is completed.

Neutral: root-tree concurrency and transport-request concurrency remain different quantities and are recorded separately. Native response quality is not scored. Dataset acquisition may still use an upstream distribution service, but a run is reproducible only after immutable revision and content-digest validation. The Python integrations remain uninvolved unless framework-specific server capabilities are later shown to affect this engine-neutral client workflow.

Use a closed AgentX source and delegate replay scheduling to release-pinned AIPerf (accepted)

Section titled “Use a closed AgentX source and delegate replay scheduling to release-pinned AIPerf (accepted)”
  • Pros: Keeps one engine-neutral Bench workflow and one native session-tree scheduler, Preserves InferLab authority for qualified inputs, resolved policy, validation, and records
  • Cons: Requires qualification of the pinned AIPerf artifact contract and upstream source snapshot

Extend the existing linear-session source to accept AgentX traces (rejected)

Section titled “Extend the existing linear-session source to accept AgentX traces (rejected)”
  • Pros: Avoids adding a third Bench source variant
  • Cons: Conflates live-response finite sessions with source-response duration replay and branching
  • Rejected because: The two workflows have different response authorities, scheduling units, completion rules, and evidence.

Build a general InferLab DAG and trace-replay runtime (rejected)

Section titled “Build a general InferLab DAG and trace-replay runtime (rejected)”
  • Pros: Could support arbitrary future workflow graphs without relying on AIPerf scheduling
  • Cons: Duplicates an existing native scheduler and introduces a public abstraction before another workflow needs it
  • Rejected because: AgentX requires one concrete replay workflow, not a general workflow language or second scheduler.

Invoke the InferenceX benchmark script as the AgentX client (rejected)

Section titled “Invoke the InferenceX benchmark script as the AgentX client (rejected)”
  • Pros: Tracks the upstream benchmark entrypoint directly
  • Cons: Makes a moving shell script own configuration, toolchain behavior, and artifacts outside InferLab records
  • Rejected because: InferLab uses AIPerf to eliminate framework and script-specific Bench wiring while retaining reproducible lifecycle evidence.