Skip to content

Decision record · ADR

ADR-0078: Run Engine-provided KV stores for the Specialized Engine under open backend names

Status: accepted | Date: 2026-11-09

References: RFC-0003, ADR-0069, ADR-0071, ADR-0075

A server’s kv_store names a backend, and the shared inferlab-kvstore package lowers two: mooncake, a master process InferLab runs, and filesystem, a bound directory InferLab does not operate (ADR-0069, ADR-0071). The adapter protocol already carries the backend as an open string, and the KV store plan already lets an integration declare the store process’s ports, readiness, and command while the control plane places, starts, and stops it. One Specialized Engine implementation ships its own store: an index its Engines join, launched with a listen address and a metrics port, serving /health and /metrics on the metrics port, opening no device, and writing no disk state. Its Engines join it with --kv-tier-store HOST:PORT and implement neither Mooncake’s client nor its protocol, so a managed server of that Engine declaring a store is refused today and cross-replica prefix sharing cannot be measured under InferLab. The Specialized Engine integration serves several Engine implementations and, under ADR-0071, keeps no Engine-specific branch.

The Specialized Engine integration accepts, beside mooncake and filesystem, any KV store backend name of lowercase ASCII letters, digits, -, and _ as an Engine-provided store, the way it accepts open KV-transfer names under ADR-0075. Such a backend requires the stack setting kv_store_command, the leading argv of the store executable, and takes no store settings: store tuning belongs in that command. The integration plans one store process InferLab runs, with the store’s listen address as its endpoint, a metrics port, and readiness on GET /health on that port, and renders <kv_store_command> --listen HOST:PORT --metrics-port PORT. The control plane runs it exactly as it runs a Mooncake master: zero devices, beside the Gateway, started and ready before any Engine, stopped after every Engine, and failing the server if it exits while the server runs. Every Engine receives --kv-tier-backend <name> --kv-tier-store HOST:PORT with the declared name unchanged, and the Engine rejects a name it does not implement before SERVING. Each Engine reports its tier state, location, access, and publication bound through the existing serve-state probe. The lowering lives in the Specialized Engine integration rather than in inferlab-kvstore, because no other integration consumes an Engine-provided store. The recorded implementation is the backend name with the integration’s version; the store software’s identity is the stack environment’s lock.

An Engine that ships its own store runs it under InferLab with the same placement, lifecycle, readiness, failure, and record as a Mooncake master, so prefix sharing across its replicas becomes measurable and qualifiable. Existing workspaces are unaffected, and no adapter protocol, record schema, or control-plane change is needed. A misspelled backend is caught at planning only when the stack lacks kv_store_command, and otherwise only when the Engine refuses the name before SERVING, after the store and Engines launch. The store’s command-line convention, --listen and --metrics-port with /health readiness, becomes part of the Specialized Engine contract, so an Engine’s store must adopt it. A store that exits fails the server even though its Engines would keep serving on local tiers, because measurements after that point would silently describe a server without its tier. A cold start depends on each Engine’s FlushCache clearing the store entries it could read, as the existing reset rule requires; InferLab adds no store restart to emulate one.

Accept open backend names in the Specialized Engine integration, each an Engine-provided store run from the stack’s kv_store_command (accepted)

Section titled “Accept open backend names in the Specialized Engine integration, each an Engine-provided store run from the stack’s kv_store_command (accepted)”
  • Pros: The record and the Engine argument carry the store’s true name, The integration keeps no Engine-specific branch, matching open KV-transfer names
  • Cons: A misspelled name is caught only when the Engine refuses it, after launch

Add a fixed engine backend value, with the name passed to the Engine coming from a stack setting (rejected)

Section titled “Add a fixed engine backend value, with the name passed to the Engine coming from a stack setting (rejected)”
  • Cons: Adds an indirection, and the declared backend no longer names the store the Engine uses

Treat the Engine’s store as externally operated infrastructure, like a filesystem tier (rejected)

Section titled “Treat the Engine’s store as externally operated infrastructure, like a filesystem tier (rejected)”
  • Pros: No store process or lifecycle in InferLab
  • Cons: The store’s state outlives runs, so a measurement’s cache is not its own and cold starts depend wholly on the Engine flush, Process lifecycle, ports, and cleanup are control-plane responsibilities left unrecorded

Let the integration declare an arbitrary store command and the argument the Engine takes for its address (rejected)

Section titled “Let the integration declare an arbitrary store command and the argument the Engine takes for its address (rejected)”
  • Cons: Turns the typed KV store component into a generic workspace-declared service process, which the KV store design deliberately excluded

Add a backend for that Engine’s store to the shared inferlab-kvstore package (rejected)

Section titled “Add a backend for that Engine’s store to the shared inferlab-kvstore package (rejected)”
  • Cons: Names one Engine in shared lowering that no other integration consumes