Skip to content

Specification · RFC

RFC-0001: Specification Overview And Authority Map

Version: 0.67.2 | Status: normative | Phase: stable Owners: @Zihua Wu


[RFC-0001:C-SUMMARY] Summary (Informative)

Section titled “[RFC-0001:C-SUMMARY] Summary (Informative) ”

InferLab runs reproducible LLM inference experiments: a versioned workspace fixes the shareable baseline, stacks own framework realizations, independently launchable servers own serving intent and cases, recipes compose one server with one workload suite, local bindings supply concrete machine placement, and every execution writes durable file-first evidence. This document is the specification entry point and maps each contract area to its owning RFC.

Authority map:

  • RFC-0002 — workspace, stack, and local-placement authority: revision and source identity, local bindings and dirty-workspace policy, Pixi lifecycle, stack realization checks, ad-hoc execution, and runtime caches.
  • RFC-0003 — server, recipe, and runtime execution: server and case composition, one resolved authority, long-running and closed-loop workflows, canonical Engine and frontend topology, per-role parallelism, independent Gateway and P/D Router backends, and fused P/D frontend binding.
  • RFC-0004 — measurements and toolchains: Eval and Bench semantics, InferLab-owned measurement toolchains, and workload profiling.
  • RFC-0005 — evidence: immutable resolved execution records, runtime evidence, and operator journal.
  • RFC-0006 — the current adapter protocol and independently distributed integration packages.
  • RFC-0007 — stack-based runtime image production, validation, and container launch facts.
  • RFC-0008 — agent plugin distribution.
  • RFC-0009 — runtime time-bound semantics: operation budgets, nested attempt bounds, measurement-case deadlines, deliberately unbounded operations, and cleanup grace.
  • RFC-0010 — view-only workspace TUI: navigation across declared definitions, current CLI operations, records, and bounded live observations.
  • RFC-0011 — public product and documentation website: content authority, current public-snapshot presentation, static validation, and GitHub Pages delivery.
  • RFC-0012 — workspace web console: token-protected local access, the workspace registry, browser views and record comparison, and workflows launched through the CLI as detached jobs.
  • This document — cross-cutting operator progress and output streams (RFC-0001:C-OPERATOR-PROGRESS), error codes (RFC-0001:C-ERROR-CODES), and license-notice retention (RFC-0001:C-LICENSE-RETENTION).

Scope: The specification covers externally observable configuration authority, command semantics, protocol boundaries, and evidence. It does not prescribe crate layout, private implementation types, arbitrary workflow graphs, registry publication, or compatibility with configuration and record schemas predating this contract.

Conformance language: MUST, MUST NOT, SHOULD, and MAY are normative keywords with their customary specification meaning across every RFC in this set.

Rationale: The contract fixes the operator capabilities proved necessary — fixed-workspace installation, direct and closed-loop server lifecycle, heterogeneous multi-role placement, measurement, images, profiling, and durable evidence — while keeping production-service supervision, provider indirection, universal workflow planning, and generic measurement dependencies out of the serving stack. Topic RFCs own and version their areas independently.

Since: v0.1.0


[RFC-0001:C-WORKSPACE-AUTHORITY] Workspace And Local Authority (Normative)

Section titled “[RFC-0001:C-WORKSPACE-AUTHORITY] Workspace And Local Authority (Normative) ”

Status: superseded Superseded by: RFC-0002:C-WORKSPACE-AUTHORITY

Moved to RFC-0002:C-WORKSPACE-AUTHORITY.

Since: v0.1.0

[RFC-0001:C-RECIPE-COMPOSITION] Recipe Composition (Normative)

Section titled “[RFC-0001:C-RECIPE-COMPOSITION] Recipe Composition (Normative) ”

Status: superseded Superseded by: RFC-0003:C-RECIPE-COMPOSITION

Moved to RFC-0003:C-RECIPE-COMPOSITION.

Since: v0.1.0

[RFC-0001:C-RUNTIME-WORKFLOWS] Runtime Workflows (Normative)

Section titled “[RFC-0001:C-RUNTIME-WORKFLOWS] Runtime Workflows (Normative) ”

Status: superseded Superseded by: RFC-0003:C-RUNTIME-WORKFLOWS

Moved to RFC-0003:C-RUNTIME-WORKFLOWS.

Since: v0.1.0

[RFC-0001:C-RESOLUTION] Resolution Dry Run And Overrides (Normative)

Section titled “[RFC-0001:C-RESOLUTION] Resolution Dry Run And Overrides (Normative) ”

Status: superseded Superseded by: RFC-0003:C-RESOLUTION

Moved to RFC-0003:C-RESOLUTION.

Since: v0.1.0

[RFC-0001:C-MEASUREMENTS] Eval And Bench Measurements (Normative)

Section titled “[RFC-0001:C-MEASUREMENTS] Eval And Bench Measurements (Normative) ”

Status: superseded Superseded by: RFC-0004:C-MEASUREMENTS

Moved to RFC-0004:C-MEASUREMENTS.

Since: v0.1.0

[RFC-0001:C-INTEGRATIONS] Framework Integration Boundary (Normative)

Section titled “[RFC-0001:C-INTEGRATIONS] Framework Integration Boundary (Normative) ”

Status: superseded Superseded by: RFC-0006:C-INTEGRATIONS

Moved to RFC-0006:C-INTEGRATIONS.

Since: v0.1.0

[RFC-0001:C-EVIDENCE] Records And Evidence (Normative)

Section titled “[RFC-0001:C-EVIDENCE] Records And Evidence (Normative) ”

Status: superseded Superseded by: RFC-0005:C-EVIDENCE

Moved to RFC-0005:C-EVIDENCE.

Since: v0.1.0

[RFC-0001:C-IMAGE-BUILD] Runtime Image Production Boundary (Normative)

Section titled “[RFC-0001:C-IMAGE-BUILD] Runtime Image Production Boundary (Normative) ”

Status: superseded Superseded by: RFC-0007:C-IMAGE-BUILD

Moved to RFC-0007:C-IMAGE-BUILD.

Since: v0.1.0

[RFC-0001:C-PIXI-ENVIRONMENT-LIFECYCLE] Pixi Environment Lifecycle (Normative)

Section titled “[RFC-0001:C-PIXI-ENVIRONMENT-LIFECYCLE] Pixi Environment Lifecycle (Normative) ”

Status: superseded Superseded by: RFC-0002:C-PIXI-ENVIRONMENT-LIFECYCLE

Moved to RFC-0002:C-PIXI-ENVIRONMENT-LIFECYCLE.

Since: v0.4.0

[RFC-0001:C-INFERLAB-TOOLCHAIN] InferLab Measurement Toolchain (Normative)

Section titled “[RFC-0001:C-INFERLAB-TOOLCHAIN] InferLab Measurement Toolchain (Normative) ”

Status: superseded Superseded by: RFC-0004:C-INFERLAB-TOOLCHAIN

Moved to RFC-0004:C-INFERLAB-TOOLCHAIN.

Since: v0.6.0

[RFC-0001:C-SERVE-PARALLELISM] Component-Aware Serve Parallelism (Normative)

Section titled “[RFC-0001:C-SERVE-PARALLELISM] Component-Aware Serve Parallelism (Normative) ”

Status: superseded Superseded by: RFC-0003:C-SERVE-PARALLELISM

Moved to RFC-0003:C-SERVE-PARALLELISM.

Since: v0.7.0

[RFC-0001:C-RUNTIME-CACHE] Runtime JIT Cache Allocation (Normative)

Section titled “[RFC-0001:C-RUNTIME-CACHE] Runtime JIT Cache Allocation (Normative) ”

Status: superseded Superseded by: RFC-0002:C-RUNTIME-CACHE

Moved to RFC-0002:C-RUNTIME-CACHE.

Since: v0.8.0

[RFC-0001:C-SERVE-TOPOLOGY] Serve Topology And Role Graph (Normative)

Section titled “[RFC-0001:C-SERVE-TOPOLOGY] Serve Topology And Role Graph (Normative) ”

Status: superseded Superseded by: RFC-0003:C-SERVE-TOPOLOGY

Moved to RFC-0003:C-SERVE-TOPOLOGY.

Since: v0.10.0

[RFC-0001:C-EXPERIMENTAL-PROXY] Routing Backends And Public Endpoint (Normative)

Section titled “[RFC-0001:C-EXPERIMENTAL-PROXY] Routing Backends And Public Endpoint (Normative) ”

Status: superseded Superseded by: RFC-0003:C-EXPERIMENTAL-PROXY

Moved to RFC-0003:C-EXPERIMENTAL-PROXY.

Since: v0.10.0

[RFC-0001:C-WORKLOAD-PROFILING] Workload-Attached Nsight Systems Profiling (Normative)

Section titled “[RFC-0001:C-WORKLOAD-PROFILING] Workload-Attached Nsight Systems Profiling (Normative) ”

Status: superseded Superseded by: RFC-0004:C-WORKLOAD-PROFILING

Moved to RFC-0004:C-WORKLOAD-PROFILING.

Since: v0.10.0

[RFC-0001:C-ENVIRONMENT-CHECKS] Environment Realization Checks (Normative)

Section titled “[RFC-0001:C-ENVIRONMENT-CHECKS] Environment Realization Checks (Normative) ”

Status: superseded Superseded by: RFC-0002:C-ENVIRONMENT-CHECKS

Moved to RFC-0002:C-ENVIRONMENT-CHECKS.

Since: v0.23.0

[RFC-0001:C-SCRATCHPAD-JOURNAL] Operator Scratchpad Journal (Normative)

Section titled “[RFC-0001:C-SCRATCHPAD-JOURNAL] Operator Scratchpad Journal (Normative) ”

Status: superseded Superseded by: RFC-0005:C-SCRATCHPAD-JOURNAL

Moved to RFC-0005:C-SCRATCHPAD-JOURNAL.

Since: v0.50.0

[RFC-0001:C-AGENT-PLUGIN] Agent Plugin Distribution (Normative)

Section titled “[RFC-0001:C-AGENT-PLUGIN] Agent Plugin Distribution (Normative) ”

Status: superseded Superseded by: RFC-0008:C-AGENT-PLUGIN

Moved to RFC-0008:C-AGENT-PLUGIN.

Since: v0.54.0

[RFC-0001:C-ERROR-CODES] Error Code Registry (Normative)

Section titled “[RFC-0001:C-ERROR-CODES] Error Code Registry (Normative) ”

InferLab reports every runtime failure under exactly one error code. When a command fails, the CLI MUST exit nonzero and MUST write one diagnostic line to stderr of the form error[<code>]: <message>. Codes are the machine-branchable contract; message text is guidance and MAY change between releases. Command-line usage errors (argument parsing) are reported by the argument parser and are outside this registry.

Registry. The leading digit groups codes by failing subsystem.

Code Meaning
E1001 Workspace discovery failed: no workspace manifest found from the start directory upward.
E1002 A required file could not be read.
E1003 Configuration could not be parsed or serialized.
E1004 Configuration is invalid: well-formed input that violates the contract.
E1005 A server override value is invalid.
E1006 A git operation failed.
E1007 The committed Pixi environment lifecycle failed: realization, launch, restoration, or an unusable environment.
E1008 The InferLab toolchain failed: unsupported platform, installation, verification, launch, or a missing toolchain for a measurement. A missing image-packaging runtime rejects the image build under E4003.
E2001 A framework integration failed: launch, transport, timeout, protocol violation, or rejection.
E3001 Placement failed: a machine binding provides fewer devices than the server requires.
E3002 Network resolution failed: a machine probe could not be launched, exited nonzero, or returned unusable evidence, or the placement has no common routable communication interface.
E4001 A closed-loop run (recipe, Bench, or image build) failed after finalizing its record; the message names the record id.
E4002 A server lifecycle operation, the built-in proxy, or profiling failed.
E4003 An image build or image selection was rejected.
E4004 A measurement data-asset acquisition or preparation operation failed.
E5001 Local record persistence failed: access, decoding, or encoding of a run, server, image, or recipe record.
E5002 A workspace operation observation could not be encoded, published, updated, or removed.
E6001 A scratchpad journal operation failed.
E7001 An agent plugin operation failed.
E8001 An ad-hoc environment execution was rejected or could not launch the operator command.
E9001 Command output could not be written or encoded.
E9002 The web console service failed: it could not generate its access token, start its runtime, or install its interruption handling; its listener could not bind or stopped; its workspace registry could not be read, parsed, or written; or a launched job’s supervisor could not start the job or record its identity or exit status.

Stability. The registry is append-only. A published code MUST keep its meaning across releases and MUST NOT be reused for a different meaning. A new failure condition MAY join an existing code only when it matches that code’s stated meaning; otherwise a new code MUST be added. Consumers MAY branch on codes and MUST NOT need to parse message text.

Since: v0.58.0

[RFC-0001:C-LICENSE-RETENTION] License Notice Retention (Normative)

Section titled “[RFC-0001:C-LICENSE-RETENTION] License Notice Retention (Normative) ”

Every distributed InferLab artifact MUST retain the project’s license notice text; a license identifier or SPDX expression alone does not satisfy retention.

Per artifact form:

  • Packages — Python wheels and sdists, and published crates — MUST contain the license file, byte-identical to the repository text, and a build or packaging gate MUST fail when a package copy drifts from that text or a built artifact lacks it.
  • The agent plugin package MUST pack the license file, and packaging MUST fail when it is absent.
  • The binary MUST embed the notice and print it with inferlab license, so a bare binary download retains it with no companion file.
  • Each release MUST publish the license as a first-class asset for human discovery.

Rationale: The license’s own condition attaches to copies of the software, so retention holds by construction in every artifact form rather than by anyone remembering to bundle a file: index installs, release downloads, plugin installs, and bare binary copies each carry the notice through the artifact itself.

Since: v0.59.0

[RFC-0001:C-OPERATOR-PROGRESS] Operator Progress And Output Streams (Normative)

Section titled “[RFC-0001:C-OPERATOR-PROGRESS] Operator Progress And Output Streams (Normative) ”

InferLab MUST preserve stdout as the command result stream. Progress MUST NOT appear on stdout for a command that returns a machine-readable or human-readable InferLab result.

Progress governed by this clause MUST be emitted as complete newline-terminated text lines on stderr. Its meaning MUST NOT depend on cursor control, animation, color, or terminal detection. Progress wording beyond the facts required by this clause is operator guidance and MAY change between releases; callers MUST use final command output and RFC-0001:C-ERROR-CODES, not progress text, for machine branching.

workspace lock, toolchain install, non-dry-run serve start, serve stop, non-dry-run recipe run, non-dry-run bench, and non-dry-run image build MUST report their first phase before waiting on a lock or starting their first potentially blocking external operation. Afterward they MUST report each applicable phase when it begins.

stack status, serve status, serve logs, scratchpad note, scratchpad show, and the dry-run forms of serve start, recipe run, bench, and image build MAY remain quiet when they complete within ten seconds. If one of those commands has not completed within ten seconds, it MUST report its current phase no later than the ten-second point and MUST report each subsequent phase when it begins.

A reported phase that remains active for ten seconds MUST emit a heartbeat no later than ten seconds after that phase was first reported. It MUST continue to emit heartbeats no more than thirty seconds apart until the phase ends. Each heartbeat MUST identify the command, the current phase, and the elapsed duration of that phase.

When a phase processes a bounded collection, progress MUST identify the current item and its ordinal position and total when those facts are known. A readiness heartbeat MUST include the most recent readiness failure when one exists. A lock-wait heartbeat MUST identify the contended lock.

The governed commands MUST expose these operator phases when they apply:

  • workspace lock: base-lock production, staged base installation, authoritative full-lock production, and restoration after failure or interruption;
  • toolchain install: installation-state inspection, writer-lock waiting, incomplete-installation replacement, Pixi installation, Eval verification, Bench verification, and image-packaging verification;
  • serve start: resolution, local and remote preflight, process launch, and readiness for each resolved process;
  • serve stop: profiler finalization, process termination, container removal, and log synchronization for each resolved process;
  • recipe run: resolution, server startup, each Eval, each Bench case or adaptive probe, profiler finalization when selected, and server cleanup;
  • bench: resolution and admission, each Bench case or adaptive probe, and profiler finalization when selected;
  • image build: resolution plus the record, package-build, assembly, inspection, export, and validation phases required by RFC-0007:C-IMAGE-BUILD;
  • scratchpad note and scratchpad show: journal-lock waiting.

When a command creates a durable record, the next progress line after creation MUST identify the record identifier and record directory, and it MUST be emitted before any subsequent potentially blocking operation. When a durable log path exists before a potentially blocking external operation starts, that operation’s phase report MUST include the log path.

Progress is ephemeral operator guidance. It MUST NOT replace authoritative execution records, command evidence, result artifacts, or failure diagnostics, and it MUST NOT be reconstructed into those artifacts as a second authority.

inferlab run MUST NOT emit wrapper progress after the operator command starts, because that command owns its stdout and stderr under RFC-0002:C-ADHOC-EXECUTION. Fast local-only commands for workspace display and license output have no progress requirement. agent install, agent update, agent uninstall, and agent doctor have no phase or heartbeat requirement under this clause; their result and error behavior remains governed by RFC-0008:C-AGENT-PLUGIN and RFC-0001:C-ERROR-CODES.

The terminal InferLab result and the error-code behavior of RFC-0001:C-ERROR-CODES remain unchanged. A progress line MUST NOT be formatted as a terminal error[<code>]: <message> diagnostic.

Rationale: Long InferLab workflows intentionally capture protocol output and preserve external-tool and experiment output in durable logs, while stdout carries one final report. Without a separate progress contract, a correctly advancing installation, readiness wait, measurement matrix, or cleanup is indistinguishable from a stalled command. Semantic phases, bounded elapsed-time heartbeats, and early record and log discovery make the operation observable without corrupting stdout, duplicating record authority, exposing protocol streams, or inventing unreliable percentages and completion estimates.

Since: v0.63.0