<id>-recipe-qualify-tp2/
The closed loop: the resolved recipe, workspace identity, and the outcome of every measurement it ran.
statusresolvedworkspacemeasurements
Experiment control plane · file-first evidence
Declare the serving shape once. Keep machine facts local. InferLab resolves the exact processes, launches them, measures, profiles, and keeps every fact in one evidence trail.
# .inferlab/workspace.d/demo.toml
[stacks.vllm]
integration = "vllm"
[servers.demo]
stack = "vllm"
model = "example"
topology = "prefill_decode"
gateway_backend = "dynamo"
pd_router_backend = "dynamo"
kv_transfer = "nixl"Every choice and status label comes from the backend support matrix; the topology and lifecycle are illustrative.
The operator loop
Four steps, one resolved execution object, no reconstructed configuration.
[servers.demo]stack = "vllm"model = "example"topology = "single"[servers.demo.cases.tp2.parallelism.outer]tensor_parallel_size = 2[recipes.qualify]server = "demo"workload_suite = "smoke-and-bench"[model_weights.example]locator = "/models/example"[machines.local]host = "127.0.0.1"devices = [0, 1]ports = [8000, 8001]"server": { "roles": [ { "kind": "serve", "replicas": [ … ] } ], "command": { "argv": [ … ], "env": { … } }, "readiness": { "kind": "http", "path": "/v1/models" }},"measurements": { "evals": [ … ], "benches": [ … ] }resolve effective commands frozenlaunch readiness verifiedmeasure eval smoke → benchrecord .inferlab/records/<id>-recipe-qualify-tp2-…/cleanup verifiedFile-first evidence
Records live in the workspace as plain files you can read, diff, and keep.
<id>-recipe-qualify-tp2/
The closed loop: the resolved recipe, workspace identity, and the outcome of every measurement it ran.
statusresolvedworkspacemeasurements<id>-serve-demo-tp2/
The server record: effective commands and environment per process, readiness evidence, and verified cleanup.
resolved.serverprocess_evidencecleanupdevice_residualsserver.stderr.log
Each process keeps its own stdout and stderr beside the record that launched it.
<id>-…-eval-000-smoke/
An Eval record: the request it sent, the response it judged, and the gate decision.
statusresolvedresult<id>-…-bench-000-throughput/
A Bench record: the frozen request population, every case, and normalized metrics beside the raw runner output.
cases[].metricscases[].completed_requestscapturecases/concurrency-000/
Per-case runner artifacts and logs, so a metric can be traced to the requests that produced it.
One experiment surface
Launch direct, Gateway-backed, disaggregated, and multi-machine serving with layered readiness, logs, and verified cleanup.
Compose correctness and performance workloads while preserving normalized metrics beside raw runner artifacts.
Attach managed Nsight Systems or engine-native trace windows to selected workloads inside the same record.
Build and validate OCI image realizations from the same versioned workspace baseline.
Watch definitions, running work, records, logs, and metrics from a view-only TUI or a local browser console.
Carry workspace revision and content identity alongside the effective execution evidence.
Start with the workspace