Decision record · ADR
ADR-0042: Overlay synthetic acceptance onto operator-owned speculative decoding configuration
Status: accepted | Date: 2026-08-30
References: RFC-0003, RFC-0006, ADR-0035, ADR-0036
Context
Section titled “Context”SemiAnalysis InferenceX requires every AgentX submission to enable speculative decoding with a synthetic acceptance behavior driven by a committed golden acceptance-length (AL) curve: one mean AL per model, thinking mode, and draft length, measured once on the SPEED-Bench coding category. The point is MLPerf-style comparability: the benchmark evaluates the inference system, not a tuned speculative head. Each framework spells the injection differently — vLLM takes synthetic rejection-sampling keys inside the --speculative-config JSON, SGLang takes SGLANG_SIMULATE_ACC_* environment variables, TensorRT-LLM takes a forced-acceptance variable with an off-by-one against the golden AL.
An earlier design draft modeled speculative decoding itself as typed framework-neutral configuration: method, draft count, and eventually draft-model locators. That coupling is wrong for two reasons. The method vocabulary is alive — InferenceX already runs mtp, eagle3, and dspark, with dflash on the horizon — and every new method would force contract and integration changes for no measurement benefit. And draft-carrying methods drag model-locator placement into the contract, although InferLab’s demonstrated need is only a controlled acceptance length with auditable provenance. Operators already express the speculative configuration itself — method, draft model, backend knobs — through framework settings and the escape hatch, and InferenceX’s own implementation confirms the split: a framework-agnostic driver decides whether to inject and resolves the AL value, then per-framework injectors rewrite the operator’s existing recipe.
The design question is therefore how InferLab owns the acceptance-length overlay — its resolution, validation, evidence, and the Eval exclusion — without owning speculative decoding configuration, while RFC-0006:C-INTEGRATIONS keeps framework-specific spelling in the integrations.
Decision
Section titled “Decision”Add a framework-neutral synthetic_acceptance declaration to server definitions and cases with exactly two forms: an explicit acceptance length, or a digest-pinned workspace-referenced golden-curve YAML resolved by model key, thinking mode, and draft count. The control plane owns curve parsing, lookup, and validation; it preserves the resolved acceptance length and curve provenance in dry-run, execution, and serve-record evidence; and it fails any Eval bound to a synthetic-acceptance server at measurement planning, because synthetic acceptance bypasses real draft-model verification.
Adapter protocol version 9 — still unpublished, so extended in place rather than re-cut — carries only the resolved acceptance length as an optional request member. Each integration overlays that length onto the operator’s existing framework speculative configuration: vLLM patches the synthetic rejection-sampling keys into the operator-supplied speculative-config JSON, SGLang and TensorRT-LLM render their environment variables including the off-by-one adjustment, and every integration fails planning with a typed error when no speculative configuration exists to overlay. Where the operator’s configuration exposes a draft-token count the integration can determine, it rejects a mismatch with the declared curve lookup key. The speculative method vocabulary, draft models, and method-specific knobs remain framework-owned and unmodeled; new methods need no InferLab change as long as the framework keeps an environment- or JSON-based acceptance mechanism.
Consequences
Section titled “Consequences”Positive: the contract stops tracking a moving method vocabulary — dspark, dflash, and future methods work through ordinary framework settings while InferLab supplies the governed acceptance overlay. Draft-model placement stays out of the serving contract entirely. The serve record states the effective acceptance length and its digest-pinned provenance, making AgentX results auditable and comparable, and correctness Evals cannot silently run against a synthetically forced verifier. The layering matches InferenceX’s own driver/injector split, so their golden YAMLs are consumed directly.
Negative: InferLab cannot verify the operator’s speculative configuration beyond overlay-target presence and draft-count mismatch checks, so a synthetically accepted run with a misconfigured method fails later at the framework rather than at planning. Hand-injected acceptance through the escape hatch without the typed declaration remains possible and ungoverned — that is the pre-existing state, not a regression, but operators mixing both paths get no evidence or Eval protection. The protocol still hard-cuts to version 9, so the adapter SDK and all integration packages move in lockstep and older integrations reject the new control plane.
Neutral: the curve’s draft-count key is a pure lookup coordinate, not a modeled serving parameter. Canonical merged workspace output preserves only the declaration, because resolution depends on the selected case. Submission policy such as mandatory speculative decoding or fixed curves remains outside InferLab: the product defines expression, validation, and evidence, not leaderboard rules.
Alternatives Considered
Section titled “Alternatives Considered”Overlay the resolved acceptance length onto operator-owned speculative configuration via protocol v9 (accepted)
Section titled “Overlay the resolved acceptance length onto operator-owned speculative configuration via protocol v9 (accepted)”- Pros: Keeps InferLab’s contract independent of the moving speculative-method vocabulary while preserving resolved-value, provenance, and evidence authority, Mirrors InferenceX’s own driver/injector split, so golden curves and framework spellings track upstream directly
- Cons: Cannot validate the operator’s speculative configuration beyond overlay-target and draft-count checks
Model speculative decoding as typed framework-neutral configuration (method, draft count, draft model) (rejected)
Section titled “Model speculative decoding as typed framework-neutral configuration (method, draft count, draft model) (rejected)”- Pros: Fully typed serving configuration with planning-time framework validation
- Cons: Couples the contract to every new method and forces draft-model locator placement, for no measurement-evidence benefit
- Rejected because: The demonstrated need is a controlled acceptance length with provenance; modeling methods and draft models is churn without owner value, and the vocabulary (mtp, eagle3, dspark, dflash) keeps moving.
Document escape-hatch injection recipes without a typed declaration (rejected)
Section titled “Document escape-hatch injection recipes without a typed declaration (rejected)”- Pros: No contract or protocol change
- Cons: No curve authority, no provenance evidence, and no protection against correctness Evals on a synthetically forced verifier
- Rejected because: Hand-written injection is the error surface the feature exists to remove; records cannot state that synthetic acceptance was in effect.
Manage golden AL curves through the measurement data-asset machinery (rejected)
Section titled “Manage golden AL curves through the measurement data-asset machinery (rejected)”- Pros: Reuses digest-pinned acquisition and evidence
- Cons: Data-asset preparation is scoped to measurement sources and runs an acquisition runtime far heavier than reading a small workspace file
- Rejected because: The curve is serve-definition reference data, not measurement source content; the workspace-relative path plus digest pattern already covers it.