Decision record · ADR
ADR-0053: Pin server processes to NUMA nodes declared on the machine binding
Status: accepted | Date: 2026-10-01
References: RFC-0002, RFC-0003
Context
Section titled “Context”On shared multi-socket hosts, managed serving measurements vary with load that other sessions place on the CPUs and memory the server happens to use. A downstream operator found engine-level A/B comparisons reliable only after binding the server to the NUMA node that hosts its GPUs. InferLab launches every server process itself and offered no way to apply such a binding. Which node is appropriate, and whether binding helps at all, is a fact about one machine and how it is shared, like its device and port pools; it is not portable server intent.
Decision
Section titled “Decision”A machine binding MAY declare numa_nodes, a nonempty list of distinct NUMA node indices on that machine. Every server process InferLab launches on the binding, engine ranks and InferLab-owned frontends alike, runs bound to those nodes for both CPU placement and memory allocation. A host process runs under numactl --cpunodebind=<nodes> --membind=<nodes> prefixed to its resolved command; a containerized process receives the container runtime cpuset for the nodes’ CPUs and memory nodes, with the CPU lists read from the launch machine during resolution, dry-run included. A host launch requires numactl on its launch machine, and every declared node must exist there and have CPUs; any such failure, or a probe that does not report every declared node, stops resolution naming the machine. The binding lives in the resolved commands, which dry-run and records already preserve, so it adds no separate record field. Omitting numa_nodes keeps today’s unbound launch.
Consequences
Section titled “Consequences”Operators on shared hosts can make managed runs as reproducible as their hand-pinned engine runs, and a record shows the binding in the command that ran. Host launches gain a numactl dependency only when a binding asks for pinning. A binding whose devices span NUMA nodes cannot pin per rank; it stays unbound until a workflow needs per-rank derivation. Measurement clients remain unpinned.
Alternatives Considered
Section titled “Alternatives Considered”Declare numa_nodes on the machine binding and bind every server process launched on it to those nodes (accepted)
Section titled “Declare numa_nodes on the machine binding and bind every server process launched on it to those nodes (accepted)”- Pros: Matches the operator’s own workflow and stays visible in configuration and in the recorded commands, Local machine facts stay in local bindings beside devices and ports
- Cons: A placement whose devices span nodes gets one binding for all its processes, so such bindings leave it unset
Derive each process binding from the NUMA affinity of its assigned GPUs behind an opt-in switch (rejected)
Section titled “Derive each process binding from the NUMA affinity of its assigned GPUs behind an opt-in switch (rejected)”- Pros: Covers placements that span nodes, one node per rank
- Cons: Adds topology probing and per-rank derivation on every launch machine for a case no workflow has asked for
Expose a free-form launch wrapper on the machine binding (rejected)
Section titled “Expose a free-form launch wrapper on the machine binding (rejected)”- Pros: Could express any launcher prefix
- Cons: Cannot bind a containerized process, whose CPUs and memory are set by the container runtime, and turns a typed fact into opaque argv