Skip to content

Decision record · ADR

ADR-0077: Name multi-host Specialized Engine options after the common distributed-launch options

Status: accepted | Date: 2026-11-07

References: ADR-0073, RFC-0003

ADR-0073 extended the Specialized Engine contract to replicas that span machines with four option names InferLab chose: --host-count, --host-index on members, --rendezvous-listen on host 0, and --rendezvous on members. The only Engine implementing multi-host serving is renaming its serving options to the names vLLM and SGLang use where the meaning is the same, and asked that the rendered multi-host options follow: --nnodes, --node-rank, and one --dist-init-addr that every node receives with the same value. The integration already sends rank zero’s allocated rendezvous address to every member, so a single flag carries the same fact; the node rank decides which side binds. The options are rendered by the Python integration and carried by no protocol field, and the Specialized Engine integration release that carries the open KV-transfer names is not yet published, so one release can carry both changes.

A replica spanning machines renders --nnodes with the replica’s rank count, --node-rank with each rank’s index (including 0 on rank zero), and --dist-init-addr with rank zero’s rendezvous address, identical on every rank, in place of the four ADR-0073 names. Rank zero binds that address and every other rank connects to it. A one-machine replica renders none of them. There is no transition: Engines switch in the same change that pins the integration release rendering the new names.

Positive: the multi-host contract reads as the common distributed-launch vocabulary, and the rendezvous fact is one flag. Negative: an Engine built against the earlier names refuses every multi-host launch from the new integration release until it switches; That Engine switches in its cutover commit, and the backend support matrix’s multi-host qualification, recorded under the earlier names, needs a new record under the new names. Neutral: everything else ADR-0073 decided — placement, per-host rendering, host 0 answering for the replica, and cleanup — is unchanged.

Render –nnodes, –node-rank, and one –dist-init-addr on every rank (accepted)

Section titled “Render –nnodes, –node-rank, and one –dist-init-addr on every rank (accepted)”
  • Pros: Operators and Engine authors read the names vLLM and SGLang already use, One rendezvous flag with the same value on every rank, with the role taken from the node rank
  • Cons: Breaks Engines built against the earlier names, which must switch with the integration release

Keep –host-count, –host-index, –rendezvous-listen, and –rendezvous (rejected)

Section titled “Keep –host-count, –host-index, –rendezvous-listen, and –rendezvous (rejected)”
  • Cons: Keeps names InferLab invented while the only implementing Engine moves its other options to the common names

Accept both spellings during a transition (rejected)

Section titled “Accept both spellings during a transition (rejected)”
  • Cons: Two spellings in the contract and a removal to schedule, for one implementing Engine that can switch in one commit