Real-Time Media ConductorArchitecture · Current architecture
LiveKit dependencies
See which paths require LiveKit, how authority reaches the provider, and what replacing it would involve.
LiveKit is optional for RMC's durable pub/sub. The current media adapter uses LiveKit. The optional multimodal-ui test harness uses that adapter for camera/screen share and OpenAI Realtime directly for browser voice; these are example client choices. RMC does not need LiveKit to create a room, publish text, replay history, or run a delegation.
This page describes the dependency boundary. Use LiveKit and voice for client calls, channel setup, and credential scope.
What LiveKit does at the media boundary
In the optional test harness, the browser publishes camera and screen-share tracks to a LiveKit room. An authorized vision processor subscribes to assigned tracks, samples frames, and returns structured observations to RMC. RMC controls who may join or publish and maintains basic provider lifecycle metadata.
LiveKit provides the WebRTC media server and track distribution. Its selective forwarding unit forwards published tracks to subscribers. That media routing is separate from the SQLite-backed room event log. See LiveKit's SFU explanation.
The current integration does not use LiveKit Agents as the reasoning runtime. Its vision service is a purpose-built subscriber, and its reasoner is reached through a separate HTTP adapter. SIP and CCaaS capabilities offered elsewhere in the LiveKit ecosystem are not automatically available through RMC's media adapter.
Locate the actual dependencies
| Component | Current dependency | What it owns |
|---|---|---|
| RMC client pub/sub | HTTP and a multiplexed WebSocket; no LiveKit SDK dependency | Durable publication, authorized delivery, reconnect |
| RMC media adapter | LiveKit JWT claims, RoomService HTTP calls, signed webhook format | Join credentials, source grants, removal, room cleanup, presence refresh |
| Test-harness browser media session | livekit-client and explicit livekit bindings |
Track publication, native reconnect events, local cleanup |
| Example vision processor | Python livekit.rtc, LiveKit participant/track identities |
Assigned track subscription and sampled-frame processing |
| Voice integration | OpenAI Realtime and WebRTC | Microphone, audio playback, voice signaling and controls |
| Reasoner integration | reasoner/0.1 over HTTP |
Requested reasoning and returned semantic result |
Inspect the server adapter, the RMC media contract, and the example processor’s track source. A client team owns its native media session and bootstrap integration.
Separate the two room lifetimes
An RMC room is the durable application workspace. A LiveKit room is the provider's media session. The adapter maps an RMC room ID to a provider room name using a prefix, rmc- by default. It uses the RMC participant ID as the provider participant identity.
Native disconnection does not erase the durable room. Conversely, RMC room closure must arrange provider cleanup; closing the SDK's event stream alone does not disconnect local tracks.
The Go media contract covers credential issuance and webhook decoding. Optional interfaces cover provider removal/deletion and authoritative presence reads. These are concrete control operations, not a general media-frame pipeline. They are declared in media.go and wired with httpapi.WithMediaAdapter in the host.
Understand setup and revocation
RMC checks current participant authority, room status, and requested channel bindings before returning a join credential. Camera and screen-share grants map to native track sources. In the scoped credential path, subscribing and data publication are disabled. The token table explains the distinct receive-only path.
Initial token validity defaults to ten minutes; configuration accepts one minute through one hour. This is an issuance setting, not a promise that every established media session ends at that timestamp. RMC's default adapter can sign a join token locally; the demo's optional automatic-agent suppression also calls LiveKit to prepare the provider room before issuance.
On revoke or room close, RMC commits local authority changes and durable cleanup intent before calling provider removal. Failed cleanup is retried. A provider error therefore does not imply that local revocation rolled back. Media tests cover this RMC behavior using test providers.
LiveKit's documented token behavior differs by deployment: Cloud supports token revocation on participant removal or permission changes; self-hosted removal does not invalidate an existing token. LiveKit can also refresh connected clients' credentials. Validate reconnect denial against the deployed provider version and configuration; initial TTL alone is not an end-to-end revocation guarantee. See LiveKit token lifecycle (reviewed September 9, 2026).
Reason about failures by path
The following describes dependency boundaries, not a measured availability guarantee.
| Failure | Direct effect | What must be verified in the product |
|---|---|---|
| LiveKit media connection fails | Camera/screen publishing and dependent vision input are disrupted | Show media failure without discarding durable chat state |
| LiveKit control API fails | Presence refresh and provider cleanup can fail; some setup modes also depend on it | Surface pending cleanup and retry without claiming revocation rolled back |
| Reasoner service fails | The requested task fails or times out | Keep interaction usable and make the failed task visible |
| RMC WebSocket drops | Durable subscriptions reconnect from processed cursors | Keep ownership separate from native tracks; test the UI's actual teardown behavior |
| RMC process or storage fails | Durable operations and provider coordination are unavailable | Define whether existing native sessions continue, degrade, or close; test it end to end |
Media uses separate connections, while voice control, authorization, and durable coordination still depend on RMC. Test whether established native sessions continue during an RMC outage in the intended deployment.
What changing the provider would involve
Using a self-hosted LiveKit deployment keeps the LiveKit API family. The operator takes responsibility for its infrastructure and deployment-specific behavior. RMC accepts a configured secure provider URL; this guide does not certify an arbitrary deployment. See LiveKit's hosting overview.
Replacing LiveKit with another media system is a broader integration change. Implement the server control contract, replace the browser track session, update bootstrap and processor assumptions, and prove source-level permissions, identity mapping, reconnect, and cleanup. The existing test harness also has provider-specific code; replacing the RMC adapter alone would not update that client.
Sending video directly to one model could suit a narrower product with no other live track consumers. That option would still need capture, consent, authorization, lifecycle, and structured outcomes. It would change how vision processors or additional participants receive media.
Keep LiveKit as the current implementation until another product requirement justifies one of these alternatives. The open question on media replacement records the decision and required evidence.