Decision record · ADR
ADR-0069: Lower external KV store backends in one shared inferlab-kvstore package
Status: accepted | Date: 2026-10-23
References: RFC-0003, ADR-0058
Context
Section titled “Context”A server’s external KV store (RFC-0003:C-SERVE-KV-STORE) runs one store process that the selected integration renders, and vLLM, SGLang, and the Specialized Engine all consume the same mooncake backend. The facts of the store process — backend name, installed version, typed settings, command, and readiness — must not differ by which integration is selected, so they need one owner shared by those integrations. ADR-0058 answered the same question for the SMG frontend with one package per backend. Mooncake is the only store backend today.
Decision
Section titled “Decision”Lower every external KV store backend in one shared inferlab-kvstore package, one module per backend, starting with inferlab_kvstore.mooncake. The package’s plan and render select the module of the backend a server declares, so integrations depend on one package and import a backend module only for the client configuration they render themselves. How each framework connects to the store and how much memory each Engine contributes stay in the integrations.
Consequences
Section titled “Consequences”Positive: one package to release, pin, and check for every store backend, and an integration gains a backend by importing its module rather than adding a dependency. Negative: a change to one backend releases the package for every consumer, and every consumer installs every backend module. If backends come to need independent release cadences or heavy distinct dependencies, splitting a backend into its own package is a later, mechanical change.
Alternatives Considered
Section titled “Alternatives Considered”One package per backend, as inferlab-gateway-smg is for SMG (rejected)
Section titled “One package per backend, as inferlab-gateway-smg is for SMG (rejected)”- Pros: Each backend releases on its own cadence and consumers install only the backends they use
- Cons: A new package to release, pin, and inventory for every backend while only one backend exists
Render the store process inside each consuming integration (rejected)
Section titled “Render the store process inside each consuming integration (rejected)”- Pros: No shared package
- Cons: Three copies of the master command and settings that must stay identical by hand, although the store must behave the same whichever integration is selected
Put store lowering in the adapter SDK (rejected)
Section titled “Put store lowering in the adapter SDK (rejected)”- Pros: No new package
- Cons: The SDK is framework-neutral and versioned with the protocol, while store flags and distribution names change with store releases
One inferlab-kvstore package with one module per backend (accepted)
Section titled “One inferlab-kvstore package with one module per backend (accepted)”- Pros: One package to release and pin; a backend is a module, not a dependency
- Cons: Backends release together and every consumer installs every backend module