Decision record · ADR
ADR-0065: Skip occupied ports observed at resolution when allocating from a port pool
Status: accepted | Date: 2026-10-14
References: RFC-0002, RFC-0005
Context
Section titled “Context”Pool allocation took a machine’s first declared ports, considering only the server being resolved. Two servers on disjoint devices of one host therefore received the same ports, and the second failed to bind — for a large model, possibly only after loading weights. A downstream workspace with roughly ten shared hosts and many concurrent experiments avoided it by declaring one machine per device group per host, each with a disjoint port block, which grew its local bindings to 97 machines and 139 placements. Resolution already reads launch-machine facts (NUMA CPU lists) during resolution and dry-run.
Decision
Section titled “Decision”Resolution treats port occupancy as an observed machine fact, as it already treats NUMA CPU lists, and allocates around it in pool order. Not-yet-terminal records of the workspace count alongside listening sockets because they cover the interval between a port’s allocation and its server binding it, and a workspace allocation lock closes the remaining race because a workspace’s pending allocations are the only ones InferLab can see. An explicit endpoint port is an operator decision, so occupancy fails it rather than moving it. RFC-0002:C-LOCAL-PLACEMENT owns the observation, allocation, lock, and failure rules; RFC-0005:C-EVIDENCE owns the recorded observation.
Consequences
Section titled “Consequences”Operators can declare one machine per host and let concurrent servers on disjoint devices share its port pool; a real run placed two vLLM servers on two GPUs of one SSH machine binding, the second took the next pool port with the first’s listening socket recorded as the holder, and both served. Every resolution and dry-run of a placement now reaches each used launch path to list its listening sockets, so an unreachable host or a missing ss fails resolution naming the machine where only machines with NUMA pinning were read before. A process outside InferLab that listens on a pool port now moves a server’s ports instead of making it fail at bind, so effective ports can differ between runs of one definition and must be read from the record, not assumed. A record left unfinalized keeps its ports reserved until it is stopped or recovered, and the evidence names it. Concurrent launches from different workspaces or users can still collide on a port neither has bound.
Alternatives Considered
Section titled “Alternatives Considered”Observe unavailable pool ports at resolution — listening sockets on the launch path and the workspace’s live server allocations — and take the first free ports in pool order, under a workspace allocation lock (accepted)
Section titled “Observe unavailable pool ports at resolution — listening sockets on the launch path and the workspace’s live server allocations — and take the first free ports in pool order, under a workspace allocation lock (accepted)”- Pros: One machine per host serves concurrent servers on disjoint devices, Adds no persistent state: occupancy is read from the machine and from existing records, Dry-run shows the ports a launch would take now
- Cons: The same definition can receive different ports on different runs; the record holds the effective ports, A port another workspace or user allocated but has not yet bound is not seen, so that race fails at launch
Lease ports in a lock-protected file on each launch machine, acquired at launch and released at cleanup (rejected)
Section titled “Lease ports in a lock-protected file on each launch machine, acquired at launch and released at cleanup (rejected)”- Pros: Closes races across workspaces and users on one machine
- Cons: Adds a second lifecycle that must be recovered after crashes and kept consistent with records and cleanup, Shared across users, needs a location every user and workspace on the machine can write
Fail resolution when a pool port is occupied, as device occupancy fails a launch (rejected)
Section titled “Fail resolution when a pool port is occupied, as device occupancy fails a launch (rejected)”- Pros: Keeps allocation identical across runs
- Cons: Leaves the demonstrated workflow broken: the operator still needs one machine and port block per device group
Document one machine per device group with disjoint port blocks (rejected)
Section titled “Document one machine per device group with disjoint port blocks (rejected)”- Pros: No product change
- Cons: Is the workaround whose machine and placement growth motivated the change
Allocate optimistically in pool order and move to the next pool port when the server fails to bind at launch (rejected)
Section titled “Allocate optimistically in pool order and move to the next pool port when the server fails to bind at launch (rejected)”- Pros: Also covers collisions with other workspaces and users
- Cons: A bind failure can surface only after a large model has loaded its weights, so each retry can cost minutes, Dry-run cannot show the ports a launch will take, and the resolved plan would change after the record is written