Skip to content

Decision record · ADR

ADR-0076: Rebuild the AIPerf integration build as upstream main plus the heartbeat-watchdog fix

Status: accepted | Date: 2026-11-07

References: ADR-0050, ADR-0061, RFC-0004

ADR-0061 pinned AIPerf 0.13.0+inferlab.3: upstream main as of 80662af with the merged AgentX port, plus ai-dynamo/aiperf#1468, the SemiAnalysisAI scenario-defaults and agentx preset commits, a finite-invariants test exemption, and the release-branch pillow fix. It rejected rebuilding on main plus #1468 alone because the scenario-defaults and preset commits were not upstream. Upstream has since merged them (#1495, #1640), the test exemption, and a newer pillow, so of the fork’s commits only #1468, still open, is not upstream. Upstream main has moved 115 commits past the build’s base, including fixes to server-metrics collection at the warmup boundary and on long runs, failing a stalled benchmark instead of waiting, chat-template input token counts, zero-step speculative acceptance records, request timestamps, and AgentX child scheduling. Upstream main now carries version 0.14.0 without a release.

The Bench toolchain pins AIPerf 0.14.0+inferlab.1: upstream main ab610ad plus ai-dynamo/aiperf#1468, built from the operator’s fork as before. The AgentX profile keeps running the agentx scenario with every preset value declared by the profile; the catalog’s InferenceX reference provenance is unchanged. Because upstream changed agentic scheduling and the measurement metrics named above, the build is adopted only after a real-hardware Bench reads its outputs and a real-hardware AgentX run requalifies the agentic catalog, which then records the new version.

Positive: the fork reduces to the one unmerged upstream fix, and Benches gain upstream’s server-metrics, stall, timestamp, and agentic fixes. Negative: the build again bases on unreleased upstream main, so later upstream changes need the same review; some metric values change across the upgrade, notably chat input token counts and speculative acceptance statistics, so cross-version comparisons must name the AIPerf version both records carry. Neutral: once #1468 merges, the next rebuild can pin an upstream release with no fork.

Rebuild as upstream main plus ai-dynamo/aiperf#1468 only (accepted)

Section titled “Rebuild as upstream main plus ai-dynamo/aiperf#1468 only (accepted)”
  • Pros: The fork carries one patch, the one upstream has not merged, Takes upstream fixes to server-metrics collection, stall handling, chat input token counts, and speculative acceptance records
  • Cons: Bases the measurement tool on unreleased upstream main again

Keep the 0.13.0+inferlab.3 build until upstream releases 0.14.0 (rejected)

Section titled “Keep the 0.13.0+inferlab.3 build until upstream releases 0.14.0 (rejected)”
  • Cons: Keeps 115 upstream commits out, including server-metrics data-loss and stall fixes, while the fork’s own commits are already upstream

Move to an upstream nightly development build without #1468 (rejected)

Section titled “Move to an upstream nightly development build without #1468 (rejected)”
  • Cons: Loses the heartbeat-watchdog teardown fix that #1468 provides and that upstream has not merged