Specification · RFC
RFC-0002: Workspace, Stack, And Local Placement Contract
Version: 0.19.0 | Status: normative | Phase: spec Owners: @Zihua Wu
1. Summary
Section titled “1. Summary”[RFC-0002:C-SUMMARY] Summary (Informative)
Section titled “[RFC-0002:C-SUMMARY] Summary (Informative) ”This RFC governs the workspace, serving stacks, and machine-local execution bindings: what one workspace revision fixes and excludes, how stacks own framework integration and Pixi realization facts, how local placements bind portable servers to concrete machines and ranks, dirty-workspace execution policy, the committed Pixi lifecycle from clean lock production to content-confirmed reuse, stack realization checks, ad-hoc execution, and per-process runtime JIT cache allocation. RFC-0001 holds the specification overview and authority map.
Since: v0.1.0
2. Specification
Section titled “2. Specification”[RFC-0002:C-WORKSPACE-AUTHORITY] Workspace And Local Authority (Normative)
Section titled “[RFC-0002:C-WORKSPACE-AUTHORITY] Workspace And Local Authority (Normative) ”A shareable execution baseline MUST be identified by one workspace revision that fixes the relevant source tree and submodule revisions, Pixi manifest and lock, framework integration package selection, reusable definitions, and recipes.
Workspace schema version 2 MUST expose models, stacks, servers, evals, benches, workload_suites, recipes, images, and external_images as its reusable definition classes. It MUST NOT expose environment, source-set, or serve-profile definition classes.
A shared model definition MUST provide logical model identity and served_name but MUST NOT contain a model-weight locator.
A stack MUST be the single workspace authority for integration, pixi_environment, ordered workspace-relative source_paths, checks, image_postprocess, and settings. A stack MAY declare no source paths. A stack’s settings are facts of the framework or Engine implementation the stack realizes, typed by its integration: the integration MUST reject a setting it does not define before launch, and MUST return the effective stack settings — every declared value unchanged, plus its own defaults — which the control plane MUST preserve in dry-run output and the server record. Server definitions, cases, and invocation overrides MUST NOT declare stack settings.
A workspace loader MUST reject an unsupported schema version, including schema version 1, before interpreting definitions. It MUST NOT partially translate or merge definitions from an unsupported schema into schema version 2.
Ordinary execution of a named server or recipe MUST NOT require the operator to restate a model, stack, server behavior, measurement definition, framework integration version, or other reusable fact already selected by the workspace definition.
The public CLI MUST accept global --workspace <dir> before the command to select the workspace root explicitly; omission uses the standard workspace discovery path.
A workspace MAY organize reusable definitions across multiple files. File organization MUST NOT create implicit behavior-affecting precedence between equivalent definitions.
Multi-file organization MUST compose the root .inferlab/workspace.toml with fragments under .inferlab/workspace.d/*.toml as a disjoint union of identifier-keyed definitions.
An identifier declared by two workspace files MUST be rejected at load with an error naming both files.
schema_version and any workspace-global scalar MUST appear only in the root workspace file; a fragment that declares one MUST be rejected at load naming the fragment.
Every path component of the workspace configuration files — the .inferlab directory, the root workspace file, the workspace.d directory, and every workspace fragment — MUST be a non-symbolic-link filesystem entry; a symbolic-link component MUST be rejected at load, because the workspace source digest records link text rather than target content.
Every component of each declared stack source path below the workspace root MUST likewise be a non-symbolic-link entry, rejected at load for the same reason.
Every symbolic link effectively present in the digested worktree — below the workspace root, outside every .git entry, outside the workspace source exclusions (the local-bindings file and the .inferlab cache, records, runtime, and scratchpads subtrees), and outside git-ignored directories, which the walk does not enter — MUST resolve within the workspace root and outside every .git entry. A digest-visible link — one the source digest records: tracked, or untracked and not git-ignored, including a link replacing a tracked entry — MUST further carry a relative target that lexically resolves to identity-covered content: not under a workspace source exclusion and not content git-ignored as judged by the target’s owning repository. A target that lexically resolves onto or through another digest-visible link is judged against its link-resolved destination, substituting digest-visible link text until no digest-visible link component remains; a digest-visible target that resolves onto or through a machine-local link MUST be rejected naming the starting link, its target, and the machine-local link, because the machine-local link’s text is outside the recorded identity and its retarget would change effective content under an unchanged digest; a substitution chain that revisits a link is a cycle and MUST be rejected naming the starting link and its target. A violating link MUST be rejected when the workspace snapshot claims source identity, naming the link path and target — and the declaring stack when the link lies under one of its source paths — because the source digest records link text rather than target content.
A git-ignored link is machine-local state no identity claim covers, so containment alone binds it, and an absolute target is judged by whether it resolves under this workspace’s root. A git-ignored directory is a machine-local tree outside the digested worktree: the walk neither enters it nor judges its contents, because the digest cannot see them and a digest-visible link targeting git-ignored content is already rejected. A digest-visible link whose target is identity-covered remains permitted, as is a dangling internal target. Tracking state otherwise affects dirtiness, not containment; resolution is lexical for relative targets, and every intermediate link is enumerated and judged on its own because the walk covers the whole digested worktree rather than only declared stack source paths.
Local bindings MUST remain separate from shareable workspace content.
InferLab MUST discover local bindings from .inferlab/local.toml under the workspace root by default.
A workflow that permits local-binding selection MUST allow an operator to select an alternate local-bindings file explicitly with --local <file>.
Local bindings MAY supply a default placement, model-weight locators, machines, devices, ordered port pools, launch access, credentials, builders, caches, artifact locations, container facts, and other machine-private facts.
Local bindings MUST NOT contain persistent overrides for behavior-affecting model, stack, server, case, recipe, Eval, Bench, workload-suite, image, or external-image settings.
An operator MAY keep machine-local definitions in fragments under .inferlab/workspace.local.d/*.toml. Local fragments are not local bindings: they MUST NOT change any committed definition and only add new identifiers. They follow the fragment rules above, including the non-symbolic-link rule for the directory and every fragment, and MAY declare only models, servers, Evals, Benches, workload suites, and recipes; one declaring a stack, image, or external image MUST be rejected at load naming the fragment. serve start, recipe run, and bench, including their dry runs, MUST compose local fragments, and only after the committed workspace has composed and validated on its own, as further members of the same disjoint union: a local definition that redeclares an identifier the committed workspace declares MUST be rejected naming both files, so no committed definition can depend on a local one. Every other workflow, including image build and run, MUST NOT compose them, so a reference there to a local definition fails as unknown. When local fragments are composed, the directory and every fragment in it MUST be git-ignored and untracked; otherwise load MUST fail naming the path and that it must be git-ignored and untracked. Local fragments are thereby outside the workspace source digest and the dirty marker, and remote machines MUST NOT need them, because definitions resolve on the controller. Execution that composes any local definition MUST state that its workspace revision alone does not reproduce it.
Long-running server and closed-loop recipe workflows MUST permit execution from a dirty workspace for local development.
Dirty-workspace resolution MUST identify the baseline revision, the effective dirty source state by content digest, and that revision-only reproduction is not possible.
The workspace serving closure MUST NOT include lm-eval, AIPerf, an InferLab Eval or Bench runner, or an Eval-only or Bench-only Pixi environment.
The InferLab-owned toolchains (RFC-0004:C-INFERLAB-TOOLCHAIN) MUST be fixed by the installed InferLab release and remain outside workspace package authority; workspace-side integration packages remain inside the workspace dependency authority (RFC-0006:C-INTEGRATIONS).
Rationale: A stack gives each framework closure one shareable owner without making machine placement or model-weight paths portable configuration. Git and Pixi provide the serving baseline shared by collaborators, while a conventional local file supplies private machine facts. Explicit schema rejection is safer than translating a removed ownership model into superficially similar fields. The containment walk covers the digested worktree rather than only declared source paths because a link outside those paths still enters source identity and can bridge into framework-consumed content. Git-ignored directories stay outside that identity because real editable installations legitimately place machine-local links there.
Since: v0.1.0
[RFC-0002:C-PIXI-ENVIRONMENT-LIFECYCLE] Pixi Stack Environment Lifecycle (Normative)
Section titled “[RFC-0002:C-PIXI-ENVIRONMENT-LIFECYCLE] Pixi Stack Environment Lifecycle (Normative) ”A stack’s local serving realization MUST be defined by the committed Pixi manifest and lock together with the Pixi environment selected by that stack.
The schema-version-2 CLI MUST NOT expose env lock or env status; workspace lock and stack status replace them.
The public CLI MUST provide workspace lock, which can produce the full workspace lock from a clean local package prefix.
workspace lock MUST NOT load local bindings or accept --local. It MUST NOT require manual intermediate manifest edits or out-of-band package installation. Successful lock production MUST preserve the authoritative workspace manifest byte-for-byte and leave the resulting full lock as the workspace lock. After a handled failure or interruption, it MUST restore the authoritative manifest and any pre-existing workspace lock.
Intermediate package state used while producing a lock MUST NOT become a second manifest, lock, stack identity, or experiment input.
Ordinary workspace setup MUST install the selected stack environments from the committed lock without updating that lock or the authoritative manifest.
Dry-run, long-running server, and closed-loop recipe workflows MUST activate the already installed Pixi environment selected by the server’s stack without installing packages or updating the manifest or lock, unless a runtime image substitutes for the local stack realization (RFC-0003:C-RUNTIME-WORKFLOWS). In that case the image is the serving realization and no local serving installation is consulted.
When no image substitutes, those workflows MUST fail before server launch or measurement execution — naming the stack, Pixi environment, and required locked installation action — when the selected environment has never been installed, when the committed lock is not up to date with the manifest, or when the installation has not been confirmed usable against the current manifest and lock content.
An installation is confirmed usable against manifest and lock content only by a workflow within this specification having observed it usable against that exact content. A workspace revision change that leaves both contents unchanged MUST NOT invalidate an existing confirmation. When either content changes, the operator MUST reinstall every affected stack environment and cause a workflow within this specification to reconfirm it before use.
The public CLI MUST provide stack status [STACK]. With a stack identifier it MUST report that stack; without one it MUST report every declared stack. Standard output MUST be one JSON array containing one report object per selected stack. Each report object MUST contain stack and pixi_environment as strings and MUST retain status as the selected Pixi environment’s confirmation state. status MUST be confirmed, never-installed, or not-usable; it MUST NOT be repurposed as overall readiness. A string diagnostics field MUST be present exactly when status is not-usable and MUST describe why that environment cannot be confirmed. A string install_command field MUST be present exactly when status is never-installed or not-usable and MUST contain the required locked installation command.
Each report object MUST additionally contain a checks object and a Boolean ready. checks MUST contain a string state and an evidence array. Its state MUST be not-declared when the stack declares no checks, skipped when checks are declared but the environment is not confirmed, passed when every declared check completed with a zero exit status, failed when the first completed nonzero check caused execution to stop, or error when an infrastructural failure prevented an attempted check from producing an exit status. These states are mutually exclusive.
Each item in checks.evidence MUST contain string fields id, realization, outcome, and output. id MUST be the declared check identifier, realization MUST be local-workspace, outcome MUST be passed or failed, and output MUST be the captured combined output. A string repair_hint MUST be present exactly when the item’s outcome is failed and that check declares one. The checks object MUST contain an error object exactly when its state is error; that object MUST contain the attempted check’s string id and string diagnostics. checks.error MUST remain distinct from completed-check evidence and from the environment-level diagnostics field.
When the selected environment is confirmed and the stack declares checks, stack status MUST attempt them against the local workspace realization in declaration order on every invocation. It MUST NOT cache check outcomes or infer them from the environment confirmation marker. It MUST stop after the first completed nonzero check or the first infrastructural failure. The evidence array MUST contain, in execution order, exactly the attempted checks for which an exit status was observed; an attempt that produced no exit status MUST appear only in checks.error. When the environment is not confirmed, stack status MUST attempt no declared check; the environment status is the reason for the skipped check state.
A selected stack is overall ready exactly when its environment status is confirmed and its check state is not-declared or passed; ready MUST encode that predicate. stack status MUST emit the report for every selected stack and exit successfully exactly when every report has ready equal to true. It MUST NOT load local bindings, accept --local, install packages, update the manifest or lock, or repair a failed realization.
Before launching any server process, long-running server and closed-loop recipe workflows MUST execute the selected stack’s declared checks against each distinct machine realization the launch consumes — the controller’s installation and each remote machine’s (RFC-0002:C-ENVIRONMENT-CHECKS). A failed check MUST fail before any process launches and present the declared repair hint. This preflight does not apply when a built image substitutes, because the image realization was checked during assembly, or when a declared external image substitutes, because the workspace makes no realization-check qualification claim for it.
Experiment execution MUST NOT create a server-specific, recipe-specific, case-specific, or source-path-specific Pixi manifest, lock, or materialized environment closure.
Dry-run output and execution records MUST identify the selected stack, its Pixi environment, and the committed manifest and lock digests.
The stack environment lifecycle is independent of the InferLab-owned toolchain lifecycle in RFC-0004:C-INFERLAB-TOOLCHAIN.
Rationale: A committed lock gives collaborators one fixed workspace installation model, while the stack identifies which locked environment realizes a framework closure. Separating lock production, locked installation, and no-sync execution keeps package mutation out of experiments. Manifest-and-lock confirmation answers whether the selected environment can be used without package mutation; declared checks answer whether the concrete local realization still satisfies stack-owned invariants, including invariants over editable native artifacts when the workspace chooses to declare them. Keeping those facts separate lets stack status expose complete readiness without making InferLab infer framework artifacts, bind confirmation to source revisions, or repair a realization.
Since: v0.1.0
[RFC-0002:C-ENVIRONMENT-CHECKS] Stack Realization Checks (Normative)
Section titled “[RFC-0002:C-ENVIRONMENT-CHECKS] Stack Realization Checks (Normative) ”A stack MAY declare realization checks and image-realization postprocess steps. Both are committed workspace content: the stack owns what its framework realization must satisfy, while InferLab owns the execution contract.
The committed Pixi lock is the single authority for package content, but each installed instance of the stack’s selected Pixi environment — the local workspace environment, each remote machine’s environment, and an image environment — is a distinct realization produced by installer tooling whose behavior the lock does not fully determine. Checks assert stack-declared invariants against a realization.
A check MUST have an id unique within its stack and a workspace-relative Python script executed with the selected Pixi environment’s Python interpreter. Exit status zero is the sole pass signal. A check MUST NOT mutate the realization it examines. Check output MUST report facts, not remedies; a check declaration MAY carry a repair hint for the operator.
One stack check set MUST serve every realization point. This specification requires execution by stack status against the local workspace realization when its selected environment is confirmed (RFC-0002:C-PIXI-ENVIRONMENT-LIFECYCLE), at image-build entry against the local stack realization and inside image assembly against the image realization (RFC-0007:C-IMAGE-BUILD), and before server launch against each distinct machine realization the launch consumes (RFC-0002:C-PIXI-ENVIRONMENT-LIFECYCLE).
A completed check with a nonzero outcome MUST fail the consuming workflow; there is no warning level. For stack status, it MUST make the selected stack not ready and the command unsuccessful after the status report is emitted. Repair responsibility follows realization ownership. The operator owns workspace realizations: a check failure there means the realization drifted, the failure MUST present the declared repair hint, and InferLab MUST NOT mutate the environment itself. The build owns an image realization: a check failure there means a systematic defect in build inputs or procedure, the failure MUST NOT present the repair hint, and the remedy is correcting the inputs and rebuilding.
An image-realization postprocess step MUST be a workspace-relative Python script executed only during image assembly, after environment installation and before the checks, as a deterministic finishing step of that realization. Its effect MUST derive only from the frozen build context and the environment’s locked closure. Postprocess steps MUST NOT execute against a workspace realization.
Every check for which InferLab observes a process exit status MUST produce completed-check evidence identifying its declared check id, stack, and realization — including machine when machine-scoped — and associating its outcome and captured output with that identifier. Evidence order MUST preserve execution order. stack status MUST include completed-check evidence in its emitted status report and MUST associate the applicable repair hint with the failed check identifier. A record-producing workflow MUST persist completed-check evidence in its workflow record.
An attempted check for which InferLab observes no exit status has no completed-check outcome and MUST NOT produce outcome-bearing evidence. Its infrastructural failure MUST be represented separately from completed-check evidence, MUST make stack status unsuccessful when it occurs there, and MUST still finalize the record of a record-producing workflow.
Rationale: Realization drift is realization-specific and cheapest to catch before device-backed work. Keeping checks with the stack gives the invariant one owner across local, remote, and image realizations, while repair remains with the owner of the mutable host or immutable build process. Running declared local checks from stack status exposes realization readiness without teaching InferLab how a framework’s editable native artifacts are built or identified.
Since: v0.1.0
[RFC-0002:C-RUNTIME-CACHE] Runtime JIT Cache Allocation (Normative)
Section titled “[RFC-0002:C-RUNTIME-CACHE] Runtime JIT Cache Allocation (Normative) ”Before final process rendering, InferLab MUST allocate one runtime JIT cache root for every concrete server process allocation.
A local machine binding MAY provide the storage root from which that process cache is allocated. When it does not, InferLab MUST use deterministic workspace-local state.
A runtime JIT cache namespace MUST include the resolved workspace source state, selected stack and Pixi environment identity, concrete machine identity, and process identity. When a built or external image substitutes for the local stack realization, the namespace MUST key on the image’s immutable identity in place of the invoking workspace source state and Pixi environment identity.
Processes assigned to different machines or different process identities MUST NOT receive the same mutable cache path, even when their storage roots refer to one shared filesystem.
The selected framework integration MUST receive the allocated process cache root with each concrete allocation during final rendering.
Framework-specific cache families, subdirectories, and environment-variable names MUST remain owned by the selected integration rather than the framework-neutral control plane.
An explicit framework environment override MAY replace an integration-provided cache environment default, and the effective value MUST remain visible in dry-run output and execution records.
Real execution MUST create the allocated cache root on the target machine before materializing launch files or spawning the process.
Dry-run output and execution records MUST preserve each process cache storage root, namespace inputs, allocated path, and effective framework cache environment values.
Rationale: Runtime-generated compiler and kernel state is necessary but unsafe under an implicit shared home directory. Per-process allocation isolates mutable state, and image-keyed namespaces ensure the software identity that generates cache entries is the identity that names them. The integration owns framework variable mapping while Rust owns allocation and evidence.
Since: v0.1.0
[RFC-0002:C-ADHOC-EXECUTION] Ad-Hoc Stack And Image Execution (Normative)
Section titled “[RFC-0002:C-ADHOC-EXECUTION] Ad-Hoc Stack And Image Execution (Normative) ”The public CLI MUST provide run, which executes one operator-supplied command inside a selected stack or image realization, attaches it to the operator’s standard streams, and exits with that command’s exit status.
Realization selection MUST offer the local realization of a named stack, the host-platform assembly of a selected image build record, or a declared external image. These selections MUST be mutually exclusive.
The schema-version-2 CLI MUST NOT accept run --environment; run --stack replaces it.
For local execution, --stack <stack> MUST select the stack. When the workspace declares exactly one stack, omission MUST select it; when it declares more than one, omission MUST fail requiring an explicit stack.
When an image build record is selected, the executed realization MUST be the stack realization captured by that record. Explicit stack selection combined with either image selection MUST be rejected.
Local execution MUST activate the already installed selected Pixi environment under RFC-0002:C-PIXI-ENVIRONMENT-LIFECYCLE, installing no packages and updating neither manifest nor lock. It MUST fail before executing the command when the environment is absent or the lock is stale. Because run produces no qualification evidence, it MUST NOT require or create content-confirmation state.
For local execution, when the workspace local bindings declare a default placement whose referenced machines all launch locally, run MUST project the placement’s device set as CUDA_VISIBLE_DEVICES, together with CUDA_DEVICE_ORDER=PCI_BUS_ID, into the command environment, unless the operator’s environment already selects devices. A default placement that references a remote-launching machine or declares no devices MUST NOT project. This projection reuses the workspace’s standing device assignment and creates no allocation; it does not make the placement selectable.
The operator command MUST execute as an executable. A command name that collides with a package-manager task MUST NOT resolve to that task.
Built-image execution MUST use the image’s activation entrypoint (RFC-0007:C-IMAGE-BUILD). External-image execution MUST follow the image’s declared entrypoint (RFC-0003:C-RUNTIME-WORKFLOWS): under replace it MUST use an explicit command override and assume nothing about the image’s entrypoint or activation; under image it MUST pass the command as the arguments of the image’s own entrypoint.
run MUST NOT execute stack realization checks, allocate workflow runtime caches, or satisfy a qualification obligation. It MUST NOT accept an alternate local-bindings selector. It MUST create an execution record only when the operator selects --record, under RFC-0005:C-EVIDENCE; without --record it MUST create none.
--record, --serve, and --model MUST be accepted only for local stack execution and MUST be rejected with an image selection.
Local execution MUST inject an InferLab context into the command environment. The context MUST contain INFERLAB_CONTEXT, whose value is the context contract version 1, and INFERLAB_WORKSPACE_ROOT, the absolute workspace root. With --record it MUST also contain INFERLAB_RECORD_ID and INFERLAB_RECORD_ARTIFACTS, an existing record-owned directory the command MAY write. With --serve <record> it MUST also contain INFERLAB_SERVE_RECORD, INFERLAB_SERVE_BASE_URL, the public endpoint’s scheme, host, and port without a path, and INFERLAB_SERVE_MODEL, the served model name. With --model <id> it MUST also contain INFERLAB_MODEL_ID and INFERLAB_MODEL_PATH, the model_weights locator resolved under RFC-0002:C-LOCAL-PLACEMENT for the local machine. InferLab MUST NOT set variables outside the INFERLAB_ namespace for this context, beyond the device projection above.
--serve MUST name a readable server record whose lifecycle state is running, and --model MUST name a declared binding with a locator for the local machine; otherwise run MUST fail before executing the command.
When the inherited environment contains INFERLAB_CONTEXT, the command runs nested inside another InferLab context: run MUST replace every inherited INFERLAB_* variable with its own context and, with --record, MUST record the inherited INFERLAB_RECORD_ID, when present, as the parent record. When the inherited environment lacks INFERLAB_CONTEXT but contains any variable this context defines, run MUST fail before executing the command, naming the variables.
The context is information, not an obligation: InferLab MUST NOT require or verify that the command reads any injected variable.
A recorded run runs the command in its own process group so an interrupt can terminate it. It MUST copy the command’s standard output and standard error to the operator’s streams and to its logs. It MUST NOT give the command the operator’s terminal as standard input, while piped or redirected standard input MUST still reach the command.
Image execution MUST apply no implicit host mount. It MUST accept explicit operator-declared host mounts and device selections; either kind combined with local stack execution MUST be rejected. Every mount source MUST be an existing absolute host path bound at the identical container path, read-only unless explicitly marked writable. An unrepresentable mount MUST fail rather than be altered.
Without an explicit device selection, image execution MUST request no devices.
The shipped operator skill MUST direct agents to run ad-hoc environment commands through this operation (RFC-0008:C-AGENT-PLUGIN) and prohibit direct invocation of binaries from a materialized Pixi prefix.
Rationale: Directly invoking a binary from a materialized prefix loses the activation contract. One sanctioned command keeps probes aligned with product execution while deliberately producing no qualification evidence. Stack selection replaces a removed environment definition, and explicit image mounts and devices avoid inventing workflow allocation facts for an ad-hoc command. Workloads such as agentic evaluation or model quantization have tool-specific inputs and outputs that InferLab does not model; injecting the facts InferLab is authoritative for, and recording what was provided, keeps the wrapper thin without asking each tool to adopt InferLab conventions, and the context marker lets a nested invocation tell inherited context from operator misuse.
Since: v0.4.0
[RFC-0002:C-LOCAL-PLACEMENT] Machine-Local Placement And Model Resolution (Normative)
Section titled “[RFC-0002:C-LOCAL-PLACEMENT] Machine-Local Placement And Model Resolution (Normative) ”Local bindings MUST define machine-private execution facts independently from workspace servers and cases.
A machine binding MUST be able to declare host, ordered devices, one ordered ports pool, and its launch, workspace, cache, and container facts. Device ordinals are the indices nvidia-smi reports, which follow PCI bus order. Every process InferLab launches on allocated devices MUST receive them as CUDA_VISIBLE_DEVICES together with CUDA_DEVICE_ORDER=PCI_BUS_ID, so it opens the devices that occupancy and residual-memory checks observed, and an integration MUST NOT render either variable. The ordered port pool replaces any distinction between a primary port and extra ports. A machine binding MAY declare kv_tier_directory, an absolute path to the directory through which the machine reaches a filesystem KV store under RFC-0003:C-SERVE-KV-STORE. A machine binding MAY declare numa_nodes, a nonempty list of distinct NUMA node indices on that machine. When it does, every server process launched on that binding MUST run bound to those nodes for CPU placement and memory allocation: a host process under a numactl CPU-node and memory binding prefixed to its resolved command, and a containerized process through the container runtime’s cpuset CPUs and memory nodes. Resolution MUST read the nodes’ CPU lists on the launch machine and MUST fail, naming the machine, when a declared node is absent there, has no CPUs, or is not reported by that read, or when a host launch’s machine provides no numactl. The resolved commands MUST carry the binding.
Resolution MUST look up each used machine’s host once, on the machine running InferLab, through the system resolver, and take the first IPv4 address it returns, or its first IPv6 address when it returns no IPv4 address; a host that is already an IP address MUST be used as written. Wherever an address and a port are written together, an IPv6 address MUST be bracketed. Every listen address, endpoint, named port, and client target rendered for that machine MUST use that address, so neither the processes InferLab launches nor the clients that check and measure them resolve the name themselves. Dry-run output and the server record MUST carry each used machine’s declared host and its resolved address. A host that does not resolve MUST fail resolution naming the machine. When a used machine’s address is loopback and any machine the server uses launches over SSH, starting the server MUST warn, naming the machine, its host, the address, and the SSH machine: a loopback address reaches only this machine, which an SSH target may or may not be.
A local placement MUST NOT reference a server or server case. Compatibility between a selected placement and the resolved server MUST be decided during resolution.
A placement MUST use exactly one of two allocation forms: ordered machine pools through machines and optional roles.<role>.machines, or explicit role-local bindings through roles.<role>. A machine-pool placement MAY declare one global pool and role-specific pools that replace the global pool for those roles.
An explicit role-local binding MUST use exactly one canonical shape. A direct shape MUST declare one machine, ordered devices, and an optional endpoint_port. For a model-serving role it represents replica zero and rank zero; for roles.gateway it binds the one frontend process and carries no replica or rank coordinates. A single multi-rank shape MUST declare only a ranks list of at least two rank bindings; it represents replica zero. A multiple-replica shape MUST declare only a replicas list of at least two replica entries. Every entry in a multiple-replica list MUST itself be either a direct shape or a multi-rank shape. Fields from different shapes MUST NOT coexist.
The position of an entry in roles.<role>.replicas MUST determine its replica index. The position of each rank binding in a ranks list MUST determine its rank index. A rank binding MUST declare one machine, ordered devices, and an optional endpoint_port. Local bindings MUST NOT require or accept authored replica or rank indices.
The control plane MUST consume the integration-planned whole-replica device count when applying a machine pool. It MUST place an entire model-serving replica on one machine as one rank. It MUST NOT silently split a replica across machines; when no eligible machine can hold the entire replica, resolution MUST require explicit replica bindings.
Every explicit model-serving rank MUST bind at least one device. The rank device counts within one explicit replica MUST collectively equal that replica’s planned device count. Explicit replica bindings MAY place one replica across multiple machines and MAY give different replicas or roles different rank shapes.
Every concrete device allocated to concurrently launched ranks on one machine MUST be declared by that machine and MUST NOT overlap another allocation. A rank’s explicit endpoint port, and every port allocated from a machine pool, MUST belong to the selected machine’s declared ordered port pool and MUST NOT overlap another allocation of the same server. Every explicit endpoint port MUST be reserved before any port is allocated from a pool, so a pool allocation never takes a port that another process of the server declares.
Before allocating ports, resolution MUST observe which ports of each used machine’s pool are unavailable on that machine’s launch path, which is either the local host or one SSH target compared by its declared target string; different spellings of one host are not unified. A port is unavailable when a TCP socket listens on it there on any local address, or when a holding record of the workspace allocated it on the same launch path. For ports, a holding record is a server record at the current server record schema whose status is starting or running, or a recipe record at the current recipe record schema whose status is running and whose server record does not yet exist; a record at an earlier schema holds nothing, because this release cannot stop it and anything it still runs is observable directly. When that observation cannot be completed on a launch path, resolution MUST fail naming the machine and the error. Allocation from a pool MUST take, in pool order, the first ports that are neither unavailable nor already allocated to the same server. When a pool lacks enough available ports, resolution MUST fail naming the machine and the unavailable ports with their holders. An explicit endpoint port that is unavailable MUST fail resolution naming the port and its holder. A holder is either a listening socket, described by its owning process name and ID when the launch path reports them to the launching user, or the holding record, described by its identifier, kind, status, and the command that stops it; each unavailable port is reported with one holder. Two launching workflows in one workspace MUST NOT be allocated the same port on one launch path: a workflow that will launch MUST hold the workspace’s allocation lock from that observation until a record carrying its allocation exists, and MUST release it when it ends without writing that record or when its process terminates; a concurrent launch MUST wait for the lock, a wait governed as unbounded and terminable by operator interruption under RFC-0009:C-OPERATION-BUDGETS, and while waiting MUST report the holder last recorded for the lock: its host, process ID, and command. Dry-run MUST perform the same observation without the lock and show the ports it would allocate at that moment. Ports another workspace or user has allocated but not yet bound are not observed; whether a launch that collides with them fails is determined by the serving framework.
Before allocating devices from a machine pool, resolution MUST likewise observe, for the machines of the role’s pool in pool order until one can hold the replica, which declared devices are unavailable on the machine’s launch path: a device that reports compute applications to the probe the launch-time device check uses under RFC-0005:C-EVIDENCE, and a device that a holding record allocated on the same launch path. For devices, a holding record is a server record at the current server record schema whose status is starting, or a recipe record held for ports above; a running server’s devices are observable through their compute applications. A device that reports compute applications MUST be reported with them as its holder, each described by process ID, process name, and reported memory; otherwise a record-allocated device MUST be reported with the holding record as described above. Pool allocation MUST take, in declared order, the first devices that are neither unavailable nor already allocated to the same server. Whether a machine can hold a replica at all is judged by its declared devices, under the explicit-binding rule above; when the pool’s machines could hold it but none has enough available devices, resolution MUST fail naming the candidate machine with the most available devices, the devices required and available, and its unavailable devices with their holders, and MUST NOT wait — including under --wait-for-devices, whose wait applies only to the launch-time check of the devices resolution assigned. Explicit role-local device bindings MUST NOT be observed or moved: they remain subject only to the launch-time device check. When the device observation cannot be completed on a launch path, resolution MUST fail naming the machine and the error. The allocation lock and the rule that two launching workflows in one workspace are not allocated the same resource cover pool devices as they cover ports, and dry-run performs the same device observation without the lock.
A machine-pool allocation MUST be deterministic from declared machine, device, and port order and the observed unavailable ports and devices. It MUST bind the derived gateway frontend process to the first eligible machine in roles.gateway.machines or the global machines pool with no devices.
An explicit-binding placement MUST declare roles.gateway whenever Gateway is derived, using the direct shape. Its binding MUST identify only that frontend process, MUST carry no replica or rank coordinates, and its device list MUST be empty. It remains a concrete zero-device process allocation with a machine and ports.
A P/D [gateway, pd_router] component binding MUST use only the roles.gateway placement. The placement MUST NOT declare roles.pd_router or duplicate the Gateway machine, ports, or process allocation.
Local bindings MAY declare recipe_placements, a table from recipe identifier to a declared placement, so a recipe’s placement stays a machine-local fact while the recipe reproduces its run without invocation flags. A placement selector MUST resolve in this order: an explicit invocation --placement, for a recipe run the local bindings’ recipe_placements entry for that recipe, the local bindings’ optional default_placement, or the sole declared placement. More than one remaining placement is ambiguous and MUST fail naming the candidates. A workflow that needs placement and resolves none MUST fail before integration rendering or external effect.
A local model_weights binding keyed by shared model identifier MAY declare fallback locator, machine_locators, or both. For each model-serving rank, resolution MUST select the locator for that rank’s machine when present, otherwise the fallback locator. A missing locator for any model-serving rank MUST fail resolution naming the model, machine, role, replica, and rank. A frontend process MUST receive no model locator. A map containing only machine-specific locators is valid.
The resolved placement MUST preserve the selected placement identity, selection source, every derived model role/replica/rank coordinate, the unavailable pool ports and pool devices observed on each used machine with what held each, every frontend process role and component binding, machine, devices, endpoint and auxiliary ports, launch facts, cache facts, and each model locator with its machine-specific or fallback source.
Rationale: Observing occupancy lets servers on disjoint devices of one host share its port pool instead of declaring one machine per device group; records cover the interval between a port’s allocation and its server binding it, and the lock closes the race inside one workspace, the only scope whose pending allocations InferLab can see. Portable server intent does not know where collaborators store weights or how their machines expose devices and ports. Whole-replica pool allocation keeps the convenient case deterministic. Role-local explicit bindings leave ordinary one-rank replicas flat, while ranks and replicas expose only the nesting demanded by multi-rank or multi-replica Engine topologies. The one closed gateway process role keeps frontend placement singular without introducing arbitrary components. Per-rank locator resolution remains necessary because machines participating in one replica may mount the same weights at different paths.
Since: v0.6.0
[RFC-0002:C-WORKSPACE-DISCOVERY] Public Workspace Discovery (Normative)
Section titled “[RFC-0002:C-WORKSPACE-DISCOVERY] Public Workspace Discovery (Normative) ”The public CLI MUST provide workspace show with an optional --json output mode.
workspace show MUST load and validate only the shareable workspace configuration. It MUST NOT load local bindings, invoke an integration, inspect a stack realization, or perform any external action.
Human-readable output MUST identify the available stacks, models, servers, server topology and cases, default case selection, recipes and their selected servers and workload suites, and the Eval, Bench, and workload-suite catalog.
JSON output MUST emit one machine-readable canonical view of the merged public workspace definition, including its schema version and identifier-keyed definition maps. Diagnostics MUST remain on stderr.
Rationale: Authors need a fast configuration and discovery path before they have machine-local placement or a framework environment. Returning the validated public authority keeps discovery distinct from dry-run, which remains the exact effective execution plan.
Since: v0.7.0