Skip to content

Decision record · ADR

ADR-0018: Observe workspaces directly with producer-owned ephemeral snapshots

Status: accepted | Date: 2026-07-16

References: RFC-0010, RFC-0002, RFC-0003, RFC-0005, RFC-0001

InferLab already has durable authorities relevant to operator observation: workspace definitions and dirty-source identity under RFC-0002, incrementally written workflow records under RFC-0003 and RFC-0005:C-EVIDENCE, and the workspace scratchpad under RFC-0005:C-SCRATCHPAD-JOURNAL. Records are created before state-affecting workflow commands, carry current lifecycle and runtime evidence, and later retain outcomes. Existing stderr progress under RFC-0001:C-OPERATOR-PROGRESS additionally exposes the current invocation phase, but it is attached to one terminal and is intentionally not durable evidence.

RFC-0010 requires one view-only TUI to combine those existing facts with current invocation progress and bounded live observations without becoming an execution surface or a second evidence authority. We must decide how CLI processes communicate phase state to that independently running reader, including before record allocation and for commands that own no record, and how the reader refreshes its view.

The TUI observes one discovered or selected workspace and performs no lifecycle or experiment actions. Workspace definitions retain their revision and dirty-source authority. In-progress and finalized records remain authoritative for resolved execution, lifecycle, runtime evidence, and outcomes. Operation state is machine-local, ephemeral, and may outlive a producer after abrupt termination. Multiple commands and even multiple controller hosts may share a workspace. The design must not require a daemon, service installation, database migration, workspace-schema change, or remote log synchronization merely to open the console.

We will build the TUI as an in-process, view-only reader of existing workspace authorities, supplemented by producer-owned ephemeral operation snapshots and bounded active observations on one fixed refresh interval because this is the smallest design that exposes cross-terminal invocation progress without moving execution or evidence ownership.

The reader will obtain workspace definitions, in-progress and finalized records, scratchpad entries, referenced logs, and artifacts from their existing authorities. Each workspace-scoped CLI producer will own one narrow atomic snapshot containing only producer and command identity, timing, phase, item progress, and known references needed to observe that invocation. The snapshot will project the same progress facts as RFC-0001:C-OPERATOR-PROGRESS and will not duplicate resolved configuration, record lifecycle, runtime evidence, normalized metrics, or outcomes. A record-producing operation will publish its record reference as soon as allocation makes it available.

Producer identity will include host, boot, PID, and process start time. The reader can therefore validate local liveness without mistaking PID reuse or reboot residue for a current producer; a producer on another host remains unavailable unless a bounded remote path validates it. Unsupported snapshot schemas remain isolated and incompatible rather than being partially interpreted.

The TUI will retain the last complete display generation while the next bounded refresh runs, so observation work cannot block navigation, search, or exit. A failed current observation will retain the last successful value as explicitly stale. Search will operate over the refreshed in-memory read model rather than a persisted index. When a selected operation ends, the reader will follow its explicit record reference when present and will never derive ownership from an identifier or filename.

The observation path remains asymmetric: CLI producers write only their own operation snapshot; the TUI reads but never cleans or repairs it. Normal producers remove their snapshot, while abrupt residue remains visible under host- and boot-scoped liveness semantics. Refreshes do not overlap, per-object probes are bounded and failure-isolated, and the interface exposes no hidden action palette. A daemon, database, event bus, record-progress expansion, or workspace-schema field is outside this decision.

  • Workspace definitions and records remain authoritative and independently inspectable, including dirty source state and in-progress lifecycle evidence.
  • The installed CLI binary is sufficient to open the console; there is no service to deploy, migrate, secure, or recover.
  • Independently launched and concurrent workspace commands can appear in one view, including pre-record, dry-run, and non-record invocations.
  • Same-host liveness can distinguish a current producer from PID reuse, a hard-killed producer, or residue from an earlier boot without fabricating an outcome for another host.
  • The same refreshed read model drives views and search, so navigation cannot drift behind the displayed state.
  • Updates can lag by up to the configured refresh interval, and slow bounded probes can extend a refresh generation. The previous generation remains interactive and observation times make that lag explicit.
  • Every workspace-scoped CLI path must participate in operation publication. A single command-boundary owner is needed to prevent command-specific formats from drifting.
  • Fixed polling consumes some resources even when nothing changes. The default cadence is intentionally modest, and probes remain bounded and object-scoped.
  • Abrupt termination can leave stale or unavailable snapshot files indefinitely. They are small and diagnostically useful; cleanup is deferred until real accumulation demonstrates a product need.
  • Another controller host cannot be declared live or stale from the local process table alone. The TUI reports it unavailable unless a bounded remote identity observation exists.
  • Operations that finish inside one interval may never be visible. Producers do not delay completion; records remain the history when a workflow writes one.
  • Opening and closing the TUI creates no observation record and no durable UI session history.
  • Commands outside workspace ownership, including global toolchain and agent operations, do not appear in a workspace console.
  • Richer comparison, trend, and framework-specific dashboards can be evaluated later without changing the authority boundary established here.

Direct read model plus producer-owned snapshots: read existing workspace definitions, in-progress and finalized records, scratchpad, and referenced logs directly; let each workspace CLI producer atomically publish only invocation and phase state not owned by those authorities; refresh on one fixed bounded interval. (accepted)

Section titled “Direct read model plus producer-owned snapshots: read existing workspace definitions, in-progress and finalized records, scratchpad, and referenced logs directly; let each workspace CLI producer atomically publish only invocation and phase state not owned by those authorities; refresh on one fixed bounded interval. (accepted)”
  • Pros: Preserves one authority for workspace and recorded facts, Needs no daemon, database, or separate lifecycle, Supports concurrent commands and detectable crash residue
  • Cons: Visible progress is limited by the refresh interval, Workspace CLI producers must publish their current phase, Abrupt termination can leave small stale files

Workspace observation daemon and event store: route CLI events, metrics, logs, and TUI queries through a long-running workspace service backed by a persistent index. (rejected)

Section titled “Workspace observation daemon and event store: route CLI events, metrics, logs, and TUI queries through a long-running workspace service backed by a persistent index. (rejected)”
  • Pros: Can provide immediate push updates and historical event queries, Centralizes collection from many producers
  • Cons: Adds service installation, startup, compatibility, and failure ownership, Creates a tempting second authority beside records and definitions, Requires database or log retention and migration policy
  • Rejected because: One-workspace view-only observation does not justify another long-running product component or evidence store.

Action-hosting TUI: launch and own serve, recipe, Bench, image, and cleanup workflows inside the terminal application so their progress is directly available in memory. (rejected)

Section titled “Action-hosting TUI: launch and own serve, recipe, Bench, image, and cleanup workflows inside the terminal application so their progress is directly available in memory. (rejected)”
  • Pros: Provides tight interactive control and immediate progress, Avoids cross-process operation publication for TUI-launched work
  • Cons: Makes the TUI another execution authority, Couples terminal lifetime and failure handling to long-running workflows, Cannot observe commands launched independently with the same semantics
  • Rejected because: The confirmed product boundary requires every actual operation to be initiated separately from the CLI.

Event-driven filesystem and telemetry subscriptions: watch local files and maintain provider-specific streams, refreshing only when a source emits a change. (rejected)

Section titled “Event-driven filesystem and telemetry subscriptions: watch local files and maintain provider-specific streams, refreshing only when a source emits a change. (rejected)”
  • Pros: Can reduce idle local reads and lower update latency, Works naturally for sources with reliable streaming APIs
  • Cons: Filesystem watch behavior differs across local and network filesystems, Process, readiness, GPU, and Prometheus observations still require independent schedules or streams, Combining source-specific callbacks makes coherent refresh generations harder to explain
  • Rejected because: A one-second fixed refresh is simpler, predictable, and adequate for the confirmed operator workflow.

Existing records and operation locks only: read in-progress records and current ownership locks, accepting coarser progress and no observation before record allocation. (rejected)

Section titled “Existing records and operation locks only: read in-progress records and current ownership locks, accepting coarser progress and no observation before record allocation. (rejected)”
  • Pros: Adds no new runtime document, Keeps lifecycle and evidence in their existing authorities
  • Cons: Cannot show resolution, dry runs, ad-hoc run, or other commands without records, Does not expose current phase, item, readiness failure, or progress-terminal log references uniformly
  • Rejected because: It leaves the demonstrated cross-terminal in-progress visibility gap for pre-record and non-record operations.

Persist phase progress in execution records: incrementally add current command phase, item, readiness failure, and lock waits to records. (rejected)

Section titled “Persist phase progress in execution records: incrementally add current command phase, item, readiness failure, and lock waits to records. (rejected)”
  • Pros: Makes one durable file carry both workflow state and command progress, Needs no separate operation reader path after a record exists
  • Cons: Turns ephemeral phase and heartbeat guidance into durable evidence, Still cannot cover pre-record work, dry runs, and commands that own no record without inventing records, Increases record rewrites for display state that does not explain the final experiment
  • Rejected because: Records already own resolved intent, lifecycle, runtime evidence, and outcomes; transient invocation progress has a narrower lifecycle and must not expand that authority.