Experiment control plane · file-first evidence

Make inference
experiments replayable.

Declare the serving shape once. Keep machine facts local. InferLab resolves the exact processes, launches them, measures, profiles, and keeps every fact in one evidence trail.

  • Declared workspace intent, versioned
  • Resolved once, inspected before launch
  • Recorded lifecycle, metrics, and cleanup

control plane · simulator

Qualifiedin the support matrix

registry KV transfer CLIENT workload GATEWAY + P/D ROUTER dynamo PREFILL vLLM GPU×N DECODE vLLM GPU×N DISCOVERY etcd
# .inferlab/workspace.d/demo.toml
[stacks.vllm]
integration = "vllm"

[servers.demo]
stack = "vllm"
model = "example"
topology = "prefill_decode"
gateway_backend = "dynamo"
pd_router_backend = "dynamo"
kv_transfer = "nixl"

Every choice and status label comes from the backend support matrix; the topology and lifecycle are illustrative.

Serving frameworks

The operator loop

Intent in. Evidence out.

Four steps, one resolved execution object, no reconstructed configuration.

.inferlab/workspace.d/demo.toml
[servers.demo]stack = "vllm"model = "example"topology = "single"[servers.demo.cases.tp2.parallelism.outer]tensor_parallel_size = 2[recipes.qualify]server = "demo"workload_suite = "smoke-and-bench"

File-first evidence

Every run explains itself.

Records live in the workspace as plain files you can read, diff, and keep.

  • workflow<id>-recipe-qualify-tp2/
  • server<id>-serve-demo-tp2/
  • logserver.stderr.log
  • eval<id>-…-eval-000-smoke/
  • bench<id>-…-bench-000-throughput/
  • artifactscases/concurrency-000/

<id>-recipe-qualify-tp2/

The closed loop: the resolved recipe, workspace identity, and the outcome of every measurement it ran.

  • status
  • resolved
  • workspace
  • measurements

One experiment surface

From server process to comparable result.

SERVE

Managed lifecycles

Launch direct, Gateway-backed, disaggregated, and multi-machine serving with layered readiness, logs, and verified cleanup.

MEASURE

Eval + bench

Compose correctness and performance workloads while preserving normalized metrics beside raw runner artifacts.

PROFILE

Systems evidence

Attach managed Nsight Systems or engine-native trace windows to selected workloads inside the same record.

PACKAGE

Runtime images

Build and validate OCI image realizations from the same versioned workspace baseline.

OBSERVE

Terminal + web consoles

Watch definitions, running work, records, logs, and metrics from a view-only TUI or a local browser console.

REPRODUCE

Source identity

Carry workspace revision and content identity alongside the effective execution evidence.

Start with the workspace

Your experiment should explain itself.