Specification · RFC
RFC-0006: Framework Integration Protocol
Version: 0.25.0 | Status: normative | Phase: spec Owners: @Zihua Wu
1. Summary
Section titled “1. Summary”[RFC-0006:C-SUMMARY] Summary (Informative)
Section titled “[RFC-0006:C-SUMMARY] Summary (Informative) ”This RFC governs framework integration protocol version 13: the boundary between InferLab and independently released integration packages, separate Engine, Gateway, and P/D Router planning, fused P/D frontend binding, concrete process rendering, one-shot operation behavior, generated wire projections, integration selection, and workspace-side package distribution. RFC-0001 holds the specification overview and authority map.
Since: v0.1.0
2. Specification
Section titled “2. Specification”[RFC-0006:C-INTEGRATIONS] Framework Integration Boundary (Normative)
Section titled “[RFC-0006:C-INTEGRATIONS] Framework Integration Boundary (Normative) ”Framework-specific server-setting validation, logical lowering, command rendering, capability and environment probes, framework readiness, and server-specific control actions MUST remain outside the framework-neutral control plane.
Reusable framework integrations MUST be independently distributable and selected through the committed workspace dependency authority.
InferLab MUST distribute the adapter SDK and framework integration packages as wheels. The adapter SDK and each framework integration MUST own its package version independently of the InferLab product and other workspace-side packages. Releasing one workspace-side package MUST NOT require changing the InferLab product version or an unrelated workspace-side package version. A workspace-side package release MUST publish that distribution wheel to the package index. After the InferLab v0.8.0 product Release has been published, a package-only publication MUST NOT create a package-scoped GitHub tag or Release.
For each InferLab product release after v0.8.0, its tagged source snapshot MUST declare one closed workspace-side package inventory. Every adapter SDK or framework integration distribution in that snapshot MUST appear exactly once in the inventory, and product-owned measurement runners and SDKs MUST NOT appear. Each inventory entry MUST resolve to exactly one package pyproject.toml; its [project].name and [project].version values MUST be the distribution identity and version selected by that snapshot. Each integration MUST declare an exact dependency on the adapter SDK package version it was built and tested against.
For each InferLab product release after v0.8.0, the GitHub Release for that product version MUST use the tag v<product-version>, where <product-version> equals the product version in the tagged source snapshot. The Release MUST attach the product binaries, installer, embedded agent plugin, and LICENSE.
For every entry in the tagged workspace-side package inventory, the Release MUST attach exactly one wheel whose distribution identity and version match that entry and whose complete bytes equal the wheel published by the package index for that identity and version. The package index MUST expose exactly one such wheel; zero or multiple matching wheels MUST fail release preparation. The Release MUST attach one checksum asset named <wheel-filename>.sha256 beside each wheel. Its contents MUST be exactly the lowercase hexadecimal SHA-256 digest of the complete wheel bytes, two ASCII spaces, the wheel filename, and one trailing newline. The computed digest MUST equal the package index reported SHA-256 digest. A missing inventory entry, package metadata, index wheel, index digest, wheel asset, checksum asset, or any identity, count, byte, or digest mismatch MUST fail release preparation before the product tag is created.
The complete set of wheel and checksum assets MUST be the Release inventory for workspace-side packages: it MUST contain every inventory entry exactly once and MUST contain no workspace-side distribution absent from the tagged inventory. Co-location under the product tag MUST NOT imply package-version equality or replace protocol-version and workspace-lock compatibility authority.
The existing v0.8.0 product tag and Release, its package-scoped tags and Releases, and every registry artifact published for that release MUST remain unchanged. They MUST NOT be deleted, moved, retagged, yanked, replaced, or have their published assets altered.
A committed workspace manifest MUST reference each workspace-side package by exact version or by an in-workspace filesystem path for development. A pinned version MAY resolve from an index or an in-workspace find-links location. Every filesystem reference MUST remain inside the workspace. The binary MUST NOT embed or materialize workspace-side integration content. A selected integration absent from the committed dependency set MUST fail before workflow effect naming the missing package.
Every integration operation MUST use a strict, versioned, operation-specific JSON request and response. Protocol version 16 is the sole contract for the Engine-role, Gateway, P/D Router, frontend-binding, placement, rendering, and measurement-definition shapes in this RFC. Package-version equality MUST NOT be required as an additional runtime compatibility condition.
An integration MUST validate protocol version before request shape. An unsupported version MUST produce an unsupported-protocol-version error naming received and supported versions. The control plane MUST surface both versions and the remedy: update workspace adapter pins and relock, or use an InferLab release that speaks the installed integration’s protocol. The control plane version output MUST state the adapter protocol version it speaks, so operators can check the correspondence before execution. Input or output of an earlier protocol version MUST NOT be partially interpreted as the current protocol version.
Protocol version 12 MUST carry the effective capture mechanism in plan_serve and render_serve requests whenever profiling is requested, the declared capture mechanism of each capture target, and the control-plane-assigned persistent trace directory on each model-rank allocation belonging to an engine-trace capture target under RFC-0004:C-WORKLOAD-PROFILING. An adapter advertising protocol version 11 or earlier MUST NOT emit or consume these members.
Protocol version 12 MUST carry the synthetic acceptance declaration in plan_serve and render_serve requests whenever the serve declaration carries synthetic acceptance under RFC-0003:C-SERVE-SYNTHETIC-ACCEPTANCE: the explicit acceptance length for the explicit form, or the digest-verified curve text with its digest, model key, and effective thinking mode for the curve form. The plan response MUST return the effective acceptance length when the declaration is present. An adapter advertising protocol version 11 or earlier MUST NOT emit or consume these members.
Protocol version 12 MUST carry the auxiliary model declaration in plan_serve and render_serve requests whenever the serve declaration carries auxiliary models under RFC-0003:C-SERVE-AUXILIARY-MODELS: planning receives each auxiliary identity as its kind with its logical model object (id and served_name), and each model-rank rendering allocation carries the machine-resolved locator for every declared kind. An adapter advertising protocol version 11 or earlier MUST NOT emit or consume these members.
Protocol version 12 MUST carry the smoke vision declaration in the Eval measurement-client request’s definition input under RFC-0004:C-MEASUREMENTS, and the effective image decoration of a random Bench request source — count, fixed dimensions, source form, and the directory source’s declared digest binding and sampling policy — in the Bench measurement-client request’s definition input and in Bench population-preparation inputs under RFC-0004:C-BENCH-REQUEST-SOURCES. A client advertising protocol version 11 or earlier MUST NOT emit or consume these members.
Protocol version 12 MUST carry the discovery requirement and [discovery] allocation, the registry-membership readiness probe, the HTTP action success predicate, per-target prefix-cache reset declarations defined in this clause, and the tcp target endpoint scheme for targets reached over a framework’s own TCP request plane. An adapter advertising protocol version 11 or earlier MUST NOT emit or consume these members.
Protocol version 13 MUST allow the expected value of an HTTP action success predicate to be a JSON boolean as well as a JSON string, and protocol version 14 MUST also allow a JSON integer. An adapter advertising protocol version 12 or earlier MUST NOT emit a boolean expected value.
Protocol version 14 MUST carry the resolved gateway_settings and pd_router_settings in the plan_serve request, a target-registry readiness probe whose expected role values and bootstrap port apply per canonical role kind, and each model-serving role’s resolved devices in the plan_serve request when declared under RFC-0003:C-SERVE-PARALLELISM, and for such a role the returned whole-replica device_count MUST equal it. An adapter advertising protocol version 13 or earlier MUST NOT emit or consume this member. render_serve allocations do not carry devices; each model-rank allocation’s ordered devices already records the placed devices.
Protocol version 14 MUST carry the stack’s declared stack_settings under RFC-0002:C-WORKSPACE-AUTHORITY in the plan_serve request. The plan result MUST return the complete effective stack_settings, in which every declared entry reappears unchanged, and render_serve MUST receive those effective stack settings. An integration that defines no stack setting MUST reject every declared entry.
Protocol version 14 MUST carry the resolved kv_store declaration — backend and settings — in the plan_serve request when a server declares one under RFC-0003:C-SERVE-KV-STORE. The plan result MUST return a KV store plan exactly when the request carries a declaration, and every declared store setting MUST reappear unchanged in that plan’s effective settings. A KV store plan MUST carry its per-role store access under RFC-0003:C-SERVE-KV-STORE, naming only planned model-serving roles, and MAY carry its publication bound in milliseconds. The plan result MAY declare serve_state_probe only together with a KV store plan that declares the access of at least one role.
Protocol version 14 adds the probe_serve operation for a plan that declares serve_state_probe. Its request MUST carry the resolved model, topology, the probe’s attempt bound in seconds, and every rank-zero process of a role whose store access the plan declares, with its role, replica, and endpoint. Its response MUST report each of those processes exactly once with whether it is engaged and the framework-reported state values that decided it, or a typed error naming the process when the state could not be read. The integration MUST perform the read itself within the attempt bound and MUST NOT start, stop, or signal any process. The control plane decides when to probe and what an answer means, and MUST NOT interpret the reported state values. For a KV store that the control plane runs, render_serve MUST then receive one kv_store allocation carrying its process, machine, empty devices, endpoint, ports, cache, launch, and the KV store plan, every model-rank allocation MUST carry the store’s endpoint, and the integration MUST render exactly one kv_store process for the store allocation.
Protocol version 16 carries every KV-transfer mechanism, in requests and in role links, as the mechanism name under RFC-0003:C-SERVE-TOPOLOGY rather than as a closed enumeration; the control plane MUST NOT require a name to be known to it, and each integration validates the names it accepts. Protocol version 15 and later MUST allow a KV store plan to declare itself externally_operated, for a store such as a filesystem tier that the control plane runs no process for, and its access access_observed, in which case the plan MUST carry no per-role access and no publication bound and the plan result MUST declare serve_state_probe. For an externally operated store, render_serve MUST receive no kv_store allocation, and every model-rank allocation MUST carry its machine’s bound kv_tier_directory in place of a store endpoint. A probe_serve request for a plan with observed access MUST carry the rank-zero process of every model-serving role with its kv_tier_directory when it has one, and each process’s answer MAY carry the typed members access, read_write or write, and publication_bound_ms, a non-negative integer; the control plane MUST consume these two members as the observed store access and publication bound under RFC-0003:C-SERVE-KV-STORE and continues not to interpret the remaining reported state values. A target-registry readiness probe MAY declare a JSON pointer into each registry worker entry at which every target reports its data-parallel rank count as a positive integer or its decimal string; readiness MUST NOT complete while a target’s value is absent or invalid, and the control plane MUST record each target’s value. An adapter advertising protocol version 14 or earlier MUST NOT emit or consume these members.
A local one-shot operation MUST read one complete JSON request from stdin, write one complete JSON response and no other content to stdout, and write diagnostics to stderr.
Dedicated Rust wire DTOs MUST be the sole editable source of protocol representation. InferLab MUST generate a pinned Draft 2020-12 JSON Schema from the Rust deserialization contract and generate Pydantic v2 models from that schema. Generated schema and models MUST identify their source and MUST be checked for staleness. They MUST NOT be edited independently.
Framework settings MUST cross as structured data. Unknown fields MUST be rejected. Open framework parameters are allowed only in an explicitly named settings escape hatch, whose exact effective contents MUST be returned and recorded.
plan_serve MUST receive a logical model object containing exactly id and served_name. It MUST NOT receive a model-weight locator, machine, device, rank, port, cache path, or other placement fact.
plan_serve MUST receive the topology’s requested canonical model-serving roles, replica counts, declared stack settings, per-role settings, per-role parallelism patches and resolved devices when declared, independently selected gateway_backend and pd_router_backend values when applicable, KV-transfer selection when applicable, readiness inputs, and, when profiling is requested, the effective capture mechanism selected under RFC-0004:C-WORKLOAD-PROFILING.
Protocol-v9 frontend effective settings MUST be derived by the selected integration from the selected backend identities, topology, target and endpoint requirements, other explicitly named plan_serve inputs, and versioned integration defaults. Common Engine settings and per-model-role settings MUST NOT feed Gateway or P/D Router settings. A server, its cases, and invocation-scoped overrides MAY declare gateway_settings, a table of Gateway settings that resolves as a layered patch like other settings, only when the topology has a Gateway. The integration MUST validate it against the selected Gateway’s typed settings, reject an unknown or unsupported entry before launch, and derive the Gateway’s effective settings from it together with the inputs above; every declared entry MUST appear unchanged in the returned Gateway effective settings. A server, its cases, and invocation-scoped overrides MAY likewise declare pd_router_settings, a table of P/D Router settings that resolves as a layered patch, only when the topology has a P/D Router. The integration MUST validate it against the selected P/D Router’s typed settings and reject an unknown or unsupported entry before launch; every declared entry MUST appear unchanged in the returned P/D Router effective settings, and the returned routing policies MUST follow from them.
plan_serve MUST preserve every requested model-serving role identity and replica count. It MUST NOT add, remove, rename, or resize those roles. It MUST preserve each requested frontend backend identity and MUST NOT infer an omitted backend or replace one backend selection from the other.
plan_serve MUST return the observed framework identity and version and, for every Engine role, its canonical identity, declared and effective replica count, complete effective settings, complete effective parallelism — or, from protocol version 15, none for a role whose declared devices it returns as the whole-replica device count and whose Engines report their data-parallel rank count at readiness — whole-replica device_count, required ports and endpoint kinds, readiness, profiler controls with each capture target’s declared capture mechanism, server-control actions, directed links, and declared render inputs when applicable.
Each public endpoint contract returned for a direct Engine or Gateway MUST carry an optional server_metrics member. When present, server_metrics MUST be an object containing exactly the required string member path and the optional string member port. Omission of port MUST select the public serving port. Presence of port MUST select the named port with that exact name on the concrete process that owns the public workload endpoint: the Gateway process when Gateway applies, or the rank-zero process of the direct public Engine replica. The selected name MUST be non-empty and MUST match exactly one port declared by that process. An empty, duplicate, unknown, or other-process name MUST be rejected before allocation. Presence of server_metrics MUST mean that the selected integration and topology expose an AIPerf-compatible endpoint on the public endpoint’s protocol and host. path MUST be an absolute URL path and MUST NOT contain a scheme, authority, query, fragment, credential, concrete host, or allocated port. Omission of server_metrics MUST mean that InferLab has no qualified server-metrics capability for that endpoint.
The integration MUST own the framework-specific path and logical port selection. After endpoint allocation, the control plane MUST derive the exact metrics URL once from the accepted public protocol and host, selected public or named port, and declared path. The immutable resolved execution object MUST preserve path, the optional logical port selection, and that exact URL. Measurement execution MUST consume the frozen URL without reconstructing it. A Bench request for server metrics MUST NOT cause the control plane to guess /metrics, probe alternative paths or ports, or infer support from framework identity. Protocol version 12 MUST carry the optional server_metrics object through the generated Rust and Python projections and endpoint validation. Record projection is governed by RFC-0005:C-EVIDENCE, and measurement projection is governed by RFC-0004:C-MEASUREMENTS.
Each profiling capture target that advertises framework-range window control under RFC-0004:C-WORKLOAD-PROFILING MUST declare one capture-window control binding.
The binding MUST select exactly one logical HTTP endpoint shared by its start and stop actions: the entry endpoint of that target replica or the separately planned Gateway endpoint.
The binding MUST declare its start and stop actions as typed HTTP actions.
Each capture-window HTTP action MUST declare method and path. method MUST be exactly post. path MUST be an absolute origin path beginning with one / and MUST NOT contain a scheme, authority, query, fragment, credential, concrete host, allocated port, backslash, ASCII control character, or ASCII whitespace. The control plane MUST reject an invalid action before accepting the plan and MUST compose the effective URL from the accepted logical endpoint and declared path.
A capture-window HTTP action MAY carry one optional body member. When present, body MUST be a JSON object whose complete contents are declared by the integration. The control plane MUST serialize that exact object as the request payload and send Content-Type: application/json. An absent body MUST send no request payload and MUST NOT be inferred as an empty object.
A capture-window HTTP action and a prefix-cache reset action MAY carry one optional success member; a prefix-cache conditioning action MUST NOT carry one containing a JSON Pointer and an expected JSON string or boolean value. When success is present, the action MUST succeed only when a 2xx response was received, its body parses as JSON, and the value at the pointer equals the expected value with the same JSON type; any other outcome MUST fail the action and MUST preserve the observed status and value. When success is absent, the action’s existing success rule applies unchanged. A predicate mismatch on a window-closing action is a window-closing control failure under RFC-0004:C-WORKLOAD-PROFILING.
Framework-specific window-control inputs MUST remain in that typed action. The control plane MUST execute the declared action without inferring a framework default or branching on framework identity. It MUST apply the profiling deadline owned by RFC-0004:C-WORKLOAD-PROFILING, MUST NOT follow redirects, and MUST treat only a received 2xx response as success, further constrained by the action’s success predicate when one is declared. Serialization, transport, deadline, invalid-response, redirect, and other non-2xx outcomes MUST fail the attempted action.
The control plane MUST reject a plan response that selects a Gateway endpoint when the plan has no Gateway before accepting that response.
The control plane MUST resolve the selected logical endpoint only after frontend composition and model-rank decomposition.
Dry-run output MUST preserve the declared endpoint selection, typed actions including body presence and contents, resolved control process, and effective action URLs. Execution evidence is governed by RFC-0005:C-EVIDENCE.
The capture target process and capture-window control process MAY differ.
An integration MUST NOT return a concrete process identity, host, port, or URL as the logical capture-window endpoint.
Protocol version 12 MUST carry this required binding and each action’s optional body member without a protocol-9 compatibility interpretation.
When Gateway applies, plan_serve MUST return one separate Gateway result containing the selected backend identity, implementation identity and observable version, complete effective settings, public endpoint contract, public readiness, public server controls, target requirements, render inputs, and render_source. When P/D Router applies, it MUST return one separate P/D Router result containing the selected backend identity, implementation identity and observable version, complete effective settings, routing policies, complete target semantics, bootstrap and registry requirements, internal handoff contract, readiness, render inputs, and render_source.
A Gateway result MAY carry one discovery requirement declaring the discovery process’s logical ports, readiness requirements, and render inputs. Its presence is the Gateway backend’s discovery requirement under RFC-0003:C-SERVE-TOPOLOGY. Its readiness MUST be an HTTP probe on the discovery process or process liveness, and a Gateway result that carries one MUST declare render_source integration. A P/D Router result and an Engine result MUST NOT carry one.
render_source MUST be exactly control_plane or integration and MUST mean only which side renders an accepted concrete frontend allocation. It MUST NOT represent provider identity, process ownership, placement authority, endpoint authority, lifecycle authority, cleanup authority, or record authority.
A direct single plan MUST omit both frontend results. A routed single plan MUST return Gateway and omit P/D Router. A prefill_decode plan MUST return both results.
For prefill_decode, both component plans MUST name the same rendering source and compatible co-rendering requirements. The control plane, not the integration response, MUST derive the closed [gateway, pd_router] binding.
plan_serve MUST NOT return a server-wide effective parallelism shape. It MUST NOT select model rank counts, concrete processes, machines, devices, model locators, endpoint values, runtime storage, or a readiness deadline.
After planning, the control plane MUST choose concrete model-rank decomposition and derive the gateway process when Gateway applies and the discovery process when the Gateway result declares a discovery requirement, then resolve placement, per-machine model locators, ports, cache paths, network facts, and process identities.
Every concrete allocation supplied to render_serve MUST be either one model-rank allocation or one frontend allocation. A model-rank allocation MUST contain process, role, replica, rank, rank_count, machine, ordered nonempty devices, required per-rank model_locator, optional endpoint, allocated ports, cache, launch, the control-plane-assigned persistent trace directory when the rank belongs to an engine-trace capture target, resolved role settings and parallelism, links and dependencies needed for rendering, and control-plane-supplied render inputs.
A frontend-side allocation MUST contain process, canonical process_role, one closed ordered components value of [gateway], [gateway, pd_router], or [discovery], machine, an empty device list, no model locator, allocated ports, cache, launch, links and dependencies needed for rendering, and control-plane-supplied render inputs. A gateway allocation MUST also contain its public endpoint and the accepted component plans. A discovery allocation MUST contain no public endpoint, one internal service endpoint, and one control-plane-assigned data directory under that machine’s runtime cache, which the control plane MUST empty immediately before the discovery process starts. The control plane MUST record the started discovery process as the directory’s owner and MUST fail the launch, without emptying the directory, while its recorded owner process still runs.
[gateway] MUST represent only a routed-single Gateway. [gateway, pd_router] MUST represent only a fused P/D frontend. [discovery] MUST represent only the discovery service declared by a Gateway result, and its process_role MUST be discovery. The component value MUST NOT be open or unordered, and the process_role of both Gateway bindings MUST be gateway.
render_serve MUST receive every model-rank allocation and every frontend allocation whose accepted render_source is integration. It MUST NOT receive a control-plane-rendered frontend allocation. A P/D frontend MUST be supplied once, only when both component plans name the same integration rendering source. render_serve MUST receive the [discovery] allocation whenever the topology contains one.
render_serve MUST return exactly one rendered process for every supplied allocation and no additional process. Each rendered model process MUST preserve process, role, replica, rank, and rank-count identity. Each rendered frontend process MUST preserve process, process-role, and component binding. A rendered process MAY return integration-rendered launch files under RFC-0006:C-LAUNCH-FILES.
The control plane MUST reject missing, duplicate, extra, rebound, or component-mismatched rendered processes and MUST validate every integration response semantically before accepting it for execution.
The control plane MUST retain authority for component composition, local and remote placement, model-rank decomposition, process and frontend lifecycle, endpoint allocation and ownership, runtime storage, profiler lifecycle, timeout, cancellation, cleanup, measurement orchestration, and authoritative records.
A framework integration MUST NOT own a long-running lifecycle, execute an ordinary Eval or Bench client, write authoritative InferLab records, or define an arbitrary task or shell-command operation. The protocol MUST NOT define LowerEval or LowerBench.
A readiness requirement MAY be a registry-membership probe. It MUST declare one absolute HTTP registry path on the process that owns the requirement; the logical port name of the model-serving process whose allocated host:port registry entries carry; and, as JSON Pointers into the registry path’s JSON response, the entry list, an optional entry filter with its expected string value, the entry’s role member with the expected value for each model-serving role, and the entry’s address member. It MAY declare one absolute model-list path with a JSON Pointer to the list of served model names. The control plane MUST derive the expected targets — every rank-zero model-serving process with its role and the allocated address of the named port — and MUST report the requirement ready only when every expected target matches at least one filtered entry whose role value equals the target’s role value and whose address member begins with the target’s allocated host:port followed by the end of the value or a /, and, when a model-list path is declared, the list contains the served model name. The control plane MUST evaluate the probe without branching on framework identity. When the requirement is not met it MUST record the last observed missing targets and the unexpected entries, meaning filtered entries that match no expected target.
A model-serving Engine role result MAY declare a per-target prefix-cache reset action on the replica entry endpoint of each of its replicas. A server has a per-target reset capability only when every model-serving role declares one. A plan that declares per-target reset on only some roles, or that declares both a public-endpoint reset action and any per-target reset, MUST be rejected before launch. Dry-run output MUST preserve each declared per-target action and its resolved URL for every replica.
Protocol-relevant records MUST preserve integration package identity, protocol version, framework identity, planning and rendering request and response digests, both frontend backend identities when present, the closed component binding when present, each rendering source, and effective rendered commands.
Rationale: Protocol version 7 keeps two-phase planning and rendering while replacing the overloaded routing-ownership union with explicit Gateway and P/D Router component plans. Independent backend identities express separate experimental choices; render_source states only the lowering boundary. The two closed frontend bindings encode routed single and the currently demonstrated fused P/D process without restoring a universal deployment graph. Generated projections keep Rust wire DTOs authoritative across languages, while exact wheel pins make the workspace revision authoritative for adapter behavior. Protocol version 8 adds the capture-mechanism selection and engine-trace trace storage so workload-attached engine profilers share the same two-phase boundary without the control plane learning framework profiler spellings. Protocol version 9 adds the synthetic acceptance overlay so framework-specific injection spelling and the keyed golden-curve lookup stay with the owning integration — where the operator’s speculative configuration supplies the draft count — while the control plane retains curve integrity, resolved-value validation, and evidence authority under RFC-0003:C-SERVE-SYNTHETIC-ACCEPTANCE. Protocol version 10 adds auxiliary model weight carriage so one launch can consume independent weight artifacts — such as a speculative decoding draft model — with per-machine resolution and typed evidence, while the artifact-role vocabulary stays governed under RFC-0003:C-SERVE-AUXILIARY-MODELS. Protocol version 11 adds the vision smoke declaration and the random-source image decoration so vision-language-model liveness and image-decorated load cross the same versioned measurement-definition boundary, while image content semantics stay with the measurement runtime. Protocol version 12 adds a discovery process a Gateway backend may require, registry-membership readiness, response-body success predicates, and per-target prefix-cache reset, so a frontend that registers workers through its own discovery service and exposes server control only on each worker stays describable without the control plane learning its spelling. Protocol version 13 widens the success predicate’s expected value to a JSON boolean because vLLM reports a declined prefix-cache reset only as success false inside an HTTP 200, which a string predicate cannot match.
A logical capture-window endpoint lets the integration state whether an Engine replica or Gateway exposes the framework action while InferLab remains authoritative for concrete process and endpoint allocation. Keeping that binding on the capture target identifies which action controls which trace without turning the Gateway into a capture target or introducing a general component graph. The optional JSON body keeps framework-specific control semantics in the integration-owned action while the control plane retains generic HTTP execution and evidence authority. Protocol version 7 was still an unreleased candidate when this binding was defined; requiring every generated projection and maintained integration to adopt it in that candidate preserved protocol-version compatibility instead of publishing incompatible protocol-v7 shapes.
Since: v0.1.0
[RFC-0006:C-LAUNCH-FILES] Integration-Rendered Launch Files (Normative)
Section titled “[RFC-0006:C-LAUNCH-FILES] Integration-Rendered Launch Files (Normative) ”plan_serve MAY declare workspace-authored UTF-8 text files that the integration requires as inputs to final rendering.
Each render-input declaration MUST identify exactly one source path.
The control plane MUST read each declared source path once and MUST supply its path, exact UTF-8 text, and SHA-256 digest to final rendering.
A relative render-input path MUST resolve from the workspace root.
An unreadable or non-UTF-8 render input MUST fail resolution before any process launches.
An integration that declares a render input MUST consume the control-plane-supplied text during final rendering and MUST NOT reopen that source path from its own execution environment.
Process-backed, built-image-backed, and external-image-backed adapter realizations MUST receive identical render-input contents for the same resolved workspace state.
An image-backed adapter realization MUST receive only the declared render-input contents and MUST NOT require a workspace or containing-directory mount solely to render a launch file.
render_serve MAY declare immutable text files required to launch a rendered process when the framework accepts required effective configuration only through a file input.
Each launch-file declaration MUST belong to exactly one rendered process.
Each launch-file declaration MUST contain the exact UTF-8 text, its SHA-256 digest, and a normalized relative path below that process’s control-plane-allocated runtime-cache root.
The SHA-256 digest MUST be computed over the file’s UTF-8 encoded bytes.
The declared relative path MUST have the form launch-files/<sha256>/<name>, where <sha256> is the complete lowercase hexadecimal content digest and <name> is one nonempty path component.
A launch-file relative path MUST NOT be absolute or escape its allocated runtime-cache root.
A rendered process invocation that consumes a launch file MUST reference the path formed from the control-plane-supplied runtime-cache root and the declared relative path.
Before accepting integration output, the control plane MUST validate the process binding, path confinement, content digest, and digest-bearing relative path of every declared launch file.
Dry-run MUST preserve each launch file’s resolved path, exact text, and content digest without materializing the file.
Before spawning a process that owns launch files, InferLab MUST materialize their exact text on that process’s launch machine.
InferLab MUST publish launch-file bytes atomically.
An existing launch-file target MAY be reused only when its bytes match the declared digest.
A published launch-file target MUST NOT be truncated or replaced.
A process MUST NOT observe launch-file bytes whose digest differs from its declaration.
A containerized substitution MUST expose each materialized launch file to the process at the same resolved path recorded in the command.
A launch-file materialization failure MUST fail that process before spawn and MUST NOT create a runtime handle for it.
Execution records MUST preserve each launch file’s resolved path, exact text, and content digest with the rendered process that consumed it.
Render-input files and integration-rendered launch files are machine-local runtime inputs. They MUST NOT enter an image build context, image contents, or another portable artifact merely because a runtime process consumes them.
Render-input and launch-file declarations in these required request and response shapes MUST use adapter protocol version 12 or later. An adapter advertising protocol version 11 or earlier MUST NOT emit or consume them.
Rationale: Some framework entry points accept dynamic effective configuration only through files whose contents depend on allocated endpoints, and some of those files must preserve operator-authored framework settings. Keeping the integration authoritative for framework-specific composition while assigning source-file access, target-machine materialization, dry-run, lifecycle ordering, and evidence to the control plane preserves the integration boundary in RFC-0006:C-INTEGRATIONS. Passing only declared source text makes rendering independent of whether the adapter runs as a workspace process or inside a serving image and avoids exposing the surrounding workspace to that image. Digest-bearing paths and atomic publication let concurrent runs safely reuse identical rendered text without exposing partial or mismatched content, while retaining the exact output text keeps file-only settings on the same evidence plane as command arguments and environment values. The existing runtime-cache root is known during final rendering and is already projected consistently across local, SSH, and containerized launches. These local runtime inputs are deliberately separate from the portable artifact boundary governed by RFC-0007.
Since: v0.5.0
[RFC-0006:C-OPENAI-ENDPOINT-CONTRACT] OpenAI Endpoint Contract (Normative)
Section titled “[RFC-0006:C-OPENAI-ENDPOINT-CONTRACT] OpenAI Endpoint Contract (Normative) ”The resolved public endpoint requirement MUST contain exactly one protocol plus one named completions path and one named chat-completions path.
Both named paths MUST be absolute HTTP paths. They MUST begin with /. They MUST NOT contain a scheme, authority, query, fragment, empty path segment, . segment, or .. segment. They MUST NOT end with /.
The control plane owns both named paths: for the OpenAI-compatible serving workflows governed by RFC-0003:C-OPENAI-WORKLOAD-ROUTES, the completions path is /v1/completions and the chat-completions path is /v1/chat/completions. The plan_serve endpoint declaration MUST NOT carry path values; it carries the protocol and the integration-owned capability members (server metrics, prefix-cache actions, and cache-read representation).
The endpoint requirement MUST NOT retain a singular generic API path. A client request MUST carry both named path facts even when that client uses only one of them.
For direct single, the accepted serve Engine result MUST supply the public endpoint declaration. For every Gateway-backed topology, the Gateway result MUST supply it. A P/D Router result MUST NOT supply or replace the public endpoint.
The control plane MUST reject a plan response with a missing or extra endpoint member or a public endpoint attached to P/D Router before any serving process launches. After accepting the plan response, the resolved endpoint, every measurement-client input, dry-run output, and execution record MUST preserve both control-plane-owned path strings exactly; no later layer may replace either value.
Protocol version 12 and later MUST carry this endpoint shape in plan_serve responses, resolved client inputs, generated schema and Pydantic models, cross-language fixtures, dry-run output, and records. Protocol version 11 input or output MUST NOT be partially interpreted as protocol version 12.
Rationale: One generic path cannot be the authority for both chat generation and completions scoring. Gateway is the sole public route authority whenever present; P/D Router remains an internal orchestration component. The route paths of the OpenAI-compatible workflows are fixed product facts, so the control plane owns them and the integration declares only endpoint capabilities; the integration cannot replace control-plane-owned host, port, or path values.
Since: v0.8.1