MOQ port #97

Merged

seb merged 11 commits from issue/moq-port into main 2026-06-29 10:12:49 -07:00

Conversation 3 Commits 11 Files changed 33 +9552 -3487

seb commented 2026-06-10 14:26:19 -07:00

Collaborator

Copy link

Removes LiveKit and ports everything to Iroh+MOQ

Removes LiveKit and ports everything to Iroh+MOQ

seb added 10 commits 2026-06-10 14:26:19 -07:00

Implement sonora processing pipeline ae7cf8b817

Fix state management UI issue 51b01d0708

Various fixes c46cadbd2a

Fmt stuff f9000a4752

Port voice transport from LiveKit/WebRTC to Media-over-QUIC + iroh ... de2b376caf

Replace the LiveKit/WebRTC media stack with Media-over-QUIC (moq) over an
iroh transport, with our server acting as the relay.

Server (server/):
- Embed moq-relay (Cluster + Connection + Auth) on an iroh endpoint as a
  new services::relay module; single-node today, cluster-ready for the future.
- Mint per-channel moq-tokens (HS256) the relay verifies; track live presence
  from origin announcements.
- GetChannelToken now returns {relay_addr, moq_token, broadcast_path};
  GetVoiceChannels reports participants from relay presence.
- Drop livekit-api and the LIVEKIT_* env; add MOQ_* config.

Client (src-tauri/):
- Rewrite channels::manager to dial the relay over iroh H3/WebTransport,
  publish the mic as an Opus broadcast via moq-audio, and discover/decode
  peers via origin announcements into the existing cpal+sonora pipeline.
- Derive active-speakers from RMS; emit room stats from the QUIC connection.
- Same Tauri command/event names, so the frontend is unchanged. Drop livekit.

Infra:
- proto/channels.proto: new token response shape.
- Dockerfile: Rust 1.96 (moq needs 1.95+) plus cmake/clang for aws-lc-sys.
- Vendor upstream source as pinned git submodules under vendor/ (moq, iroh)
  for in-tree reference; exclude vendor/ from deno fmt/lint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Make the moq port boot end-to-end and surface peers reliably ... a5bc14aaa9

Five fixes plus tests that would have caught two of them.

Server (server/src/services/relay.rs)
- moq-native always constructs the Quinn backend when the `quinn` feature
  is on, and Quinn refuses to bind without a TLS cert. Seed
  `config.server.tls.generate` with a throwaway hostname so the embedded
  Quinn endpoint binds cleanly. Real traffic still flows over iroh.
- Add a spawn smoke test and a token mint+verify round-trip test, both
  guarded by a `Once`-protected `install_default` on the aws-lc-rs
  crypto provider.

Client transport (src-tauri/src/channels/manager.rs, lib.rs)
- Install the `ring` rustls CryptoProvider at process start; without it
  tonic and iroh race to install conflicting providers and panic on the
  first handshake.
- Move the iroh `Endpoint` onto `ChannelsManager` behind a `OnceCell`
  and reuse it across joins; `leave_channel` now only closes the QUIC
  connection. Endpoint binding (DERP discovery, key derivation) is
  expensive enough to be worth caching for the app lifetime.
- Add an `AtomicBool` shutdown flag plumbed into the blocking mic
  thread, set by `leave_channel`. Previously an aborted async mic task
  could strand the blocking thread parked inside `current_signal.next()`
  until the input device next yielded a sample.
- Pick the currently selected iroh `PathId` for RTT instead of always
  reading `PathId::ZERO`, so RTT stays correct across path migration
  (direct vs relay).

Frontend (app/components/sidebar/channel-group.tsx)
- The relay's presence map updates asynchronously after the broadcast
  announcement lands, so a single invalidate-on-join races ahead of it
  and renders zero participants. Poll `["channels"]` every 1.5s while
  `currentChannelId` is set so the sublist fills in once presence
  catches up.

Tooling (deno.jsonc)
- Exclude `.claude/worktrees` from fmt/lint; sibling-branch worktrees
  have their own (unrelated) sources and routinely diverge in style.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Self-host the iroh relay and make voice sessions self-healing ... 07551ca00f

Voice previously depended on iroh's public infrastructure, which is
canary-tier while 1.0 is in release candidates: clients resolved the
server's endpoint through n0 DNS and fell back to n0 canary relays for
NAT traversal. In practice (home-hosted server) this produced zombie
sessions: broadcasts announced at the relay and died ~1s later, wiping
presence for everyone, while clients sat in dead calls indefinitely.

Server:
- Embed iroh-relay in the server binary (IROH_RELAY_BIND), a plain-HTTP
  websocket service meant to sit behind a TLS reverse proxy. With
  IROH_RELAY_URL set, the endpoint runs fully self-contained: custom
  relay map, no n0 discovery, no n0 relays.
- Advertise the relay URL and the endpoint's direct addresses in
  GetChannelToken, so clients dial without any third-party discovery.
  MOQ_IROH_BIND_V4 pins the voice UDP port for router forwarding;
  MOQ_IROH_PUBLIC_ADDRS advertises the WAN address.
- Track presence as a per-DID broadcast count instead of a set, so an
  announce/unannounce reorder during reconnect can't drop a live
  participant.

Client:
- Cache the shared iroh endpoint per relay URL and rebuild it (with a
  graceful close) when the server's relay changes; dial with the full
  EndpointAddr (relay URL + direct addresses).
- Watch each call's connection for unexpected closure, clear native
  state, and emit channel_disconnected; the frontend shows
  "reconnecting" and rejoins with exponential backoff.
- Retry peer catalog subscribes (3 attempts) and bound the initial
  catalog read with a timeout so a stalled peer can't block the
  announce loop.

Tests: catalog_round_trips_through_relay_between_two_peers drives the
real client publish/subscribe path through the embedded relay on
loopback with zero external infrastructure. Token-key paths in tests
must stay relative: moq-relay's Auth URL-parses the key string first,
and absolute Windows paths (c:\...) parse as URLs.

Toolchain: the pinned moq crates declare rust-version 1.95; built and
tested with rustc 1.96.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Derive the relay URL from the public server hostname ... 5d692ef99c

Enabling the embedded iroh relay previously required setting both
IROH_RELAY_BIND and IROH_RELAY_URL, and the fallback derivation only
produced localhost. The server already knows the hostname clients use
(PUBLIC_HTTP_URL), so reuse it: with just IROH_RELAY_BIND set, the
advertised URL becomes http://<public host>:<relay port>. An explicit
IROH_RELAY_URL is now only needed when the relay sits behind a TLS
reverse proxy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Keep the mic broadcast alive for the lifetime of the publish task ... 099f01c2d8

async move blocks only capture variables they mention, so binding the
broadcast and catalog producers outside the spawned task dropped them as
soon as join_channel's setup block ended. The drop unannounced the
broadcast at the relay milliseconds after it was announced, which is why
nobody ever appeared in the participant list and peers' catalog reads
failed with "moq: cancelled". Bind them inside the async block so the
task actually owns them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add a deployment guide covering the server configuration ... d02dcec720

Document every env var the server reads (auth, database, HTTP, and the
voice/relay stack), the ports to expose, reverse-proxy considerations,
and a fresh-deployment checklist, with a pointer from the README.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

seb requested review from puregarlic 2026-06-10 14:27:03 -07:00

seb force-pushed issue/moq-port from d02dcec720 to f7b1c1387f 2026-06-10 14:29:59 -07:00 Compare

gideon reviewed 2026-06-20 21:24:39 -07:00

gideon left a comment

Contributor

Copy link

Architectural Review: MOQ Port

The transition from LiveKit to Iroh+MOQ is architecturally sound and well-executed. The implementation follows project conventions and successfully integrates a decentralized media model.

Strengths:

• PTT Isolation: The use of a separate subprocess for the keyboard listener is an excellent design choice to prevent OS-level crashes from affecting the main application.
• Processing Pipeline: The 'Fast Passthrough' path in the ensures minimal latency when enhancements are disabled.
• Lifecycle: The logic in correctly optimizes relay binding.

Recommendations for Improvement:

1. Connection Resilience: In , consider replacing the linear backoff () with an exponential backoff with jitter. This will provide better resilience during relay recovery events.
2. Endpoint Lifecycle: In , the current for closing old endpoints is a good start, but implementing a more explicit drain/close sequence would prevent potential race conditions during rapid server switches.
3. Relay Addressing: I suggest enhancing the return in to provide a prioritized list or a 'recommended' primary address to reduce initial connection latency.

Verdict: 🟢 Approve
The port is high-quality and ready for merge once the author considers the above optimizations.

### Architectural Review: MOQ Port The transition from LiveKit to Iroh+MOQ is architecturally sound and well-executed. The implementation follows project conventions and successfully integrates a decentralized media model. **Strengths:** - **PTT Isolation**: The use of a separate subprocess for the keyboard listener is an excellent design choice to prevent OS-level crashes from affecting the main application. - **Processing Pipeline**: The 'Fast Passthrough' path in the ensures minimal latency when enhancements are disabled. - **Lifecycle**: The logic in correctly optimizes relay binding. **Recommendations for Improvement:** 1. **Connection Resilience**: In , consider replacing the linear backoff () with an exponential backoff with jitter. This will provide better resilience during relay recovery events. 2. **Endpoint Lifecycle**: In , the current for closing old endpoints is a good start, but implementing a more explicit drain/close sequence would prevent potential race conditions during rapid server switches. 3. **Relay Addressing**: I suggest enhancing the return in to provide a prioritized list or a 'recommended' primary address to reduce initial connection latency. **Verdict: 🟢 Approve** The port is high-quality and ready for merge once the author considers the above optimizations.

gideon reviewed 2026-06-20 21:37:43 -07:00

gideon left a comment

Contributor

Copy link

Architectural Review: MOQ Port

The transition from LiveKit to Iroh+MOQ is architecturally sound and well-executed. The implementation follows project conventions and successfully integrates a decentralized media model.

Strengths:

• PTT Isolation: The use of a separate subprocess for the keyboard listener is an excellent design choice to prevent OS-level crashes from affecting the main application.
• Processing Pipeline: The 'Fast Passthrough' path in the AudioProcessor ensures minimal latency when enhancements are disabled.
• Lifecycle: The CachedEndpoint logic in ChannelsManager correctly optimizes relay binding.

Recommendations for Improvement:

1. Connection Resilience: In start_remote_audio_with_retry, consider replacing the linear backoff (SUBSCRIBE_RETRY_DELAY * attempt) with an exponential backoff with jitter. This will provide better resilience during relay recovery events.
2. Endpoint Lifecycle: In shared_endpoint, the current tauri::async_runtime::spawn for closing old endpoints is a good start, but implementing a more explicit drain/close sequence would prevent potential race conditions during rapid server switches.
3. Relay Addressing: I suggest enhancing the relay_direct_addrs return in get_channel_token to provide a prioritized list or a 'recommended' primary address to reduce initial connection latency.

Verdict: 🟢 Approve
The port is high-quality and ready for merge once the author considers the above optimizations.

### Architectural Review: MOQ Port The transition from LiveKit to Iroh+MOQ is architecturally sound and well-executed. The implementation follows project conventions and successfully integrates a decentralized media model. **Strengths:** - **PTT Isolation**: The use of a separate subprocess for the keyboard listener is an excellent design choice to prevent OS-level crashes from affecting the main application. - **Processing Pipeline**: The 'Fast Passthrough' path in the AudioProcessor ensures minimal latency when enhancements are disabled. - **Lifecycle**: The CachedEndpoint logic in ChannelsManager correctly optimizes relay binding. **Recommendations for Improvement:** 1. **Connection Resilience**: In start_remote_audio_with_retry, consider replacing the linear backoff (SUBSCRIBE_RETRY_DELAY * attempt) with an exponential backoff with jitter. This will provide better resilience during relay recovery events. 2. **Endpoint Lifecycle**: In shared_endpoint, the current tauri::async_runtime::spawn for closing old endpoints is a good start, but implementing a more explicit drain/close sequence would prevent potential race conditions during rapid server switches. 3. **Relay Addressing**: I suggest enhancing the relay_direct_addrs return in get_channel_token to provide a prioritized list or a 'recommended' primary address to reduce initial connection latency. **Verdict: 🟢 Approve** The port is high-quality and ready for merge once the author considers the above optimizations.

puregarlic commented 2026-06-20 21:41:56 -07:00

Owner

Copy link

For clarity, there seems to be some issue with the API review status enum, and I had Gideon attempt the review a second time. Doesn’t seem to have helped.

For clarity, there seems to be some issue with the API review status enum, and I had Gideon attempt the review a second time. Doesn’t seem to have helped.

seb added 2 commits 2026-06-24 13:10:19 -07:00

Surface aberrant microphone states and stabilise device selection ... d21c5ad6ab

The long-standing "someone goes silent, others can't hear them" bug traces
to the local mic pipeline, not the transport (it predates the MOQ port).
Make those failures visible and fix a way the wrong device gets selected.

- Add an audio-health alert layer: a capture-liveness watchdog (per-callback
  counter) that flags a stalled/disconnected microphone, an outbound
  bit-exact-silence detector that tells halted capture/processing apart from
  zero gain or an OS-muted device, and an encode-failure alert. Emitted as
  `audio_alert` and shown as a toast via a new `useAudioAlerts` hook, with the
  reason and a suggested fix, so a silently-inaudible participant finds out.
- Identify audio devices by CPAL's stable DeviceId instead of the display
  label, whose ` (N)` ordinal depends on enumeration order — a persisted
  selection could otherwise resolve to the wrong physical device.
- Replace the linear subscribe-retry backoff with exponential backoff plus
  jitter so peers recovering from a relay blip don't retry in lockstep.
- Fix server clippy lints under -D warnings (collapsible let-chains; gate a
  test-only getter behind cfg(test)).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Upgrade to iroh 1.0 ... 9ab7e944ea

iroh 1.0.0 released. It can't be bumped in isolation: web-transport-iroh
0.5 (used by both our crates and moq) hard-pins iroh =1.0.0-rc.0, and iroh
types flow across those boundaries. So move the whole stack together —
iroh/iroh-relay to 1.0, web-transport-iroh to 0.6, and the moq git rev to
main (7090bb36), which adopted web-transport-iroh 0.6.

Migrate for the moq API drift the rev bump pulls in:
- moq_native::Cluster::new now returns a Result.
- moq_mux::catalog::hang::Producer moved to moq_mux::catalog::Producer.
- moq_mux::catalog::hang::Consumer is now generic; annotate the default Extra.

Both lockfiles resolve a single iroh 1.0.0. Verified: build, clippy
-D warnings, fmt, and tests on both crates (server's end-to-end relay test
exercises the real publish/relay/subscribe path through the new stack).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

seb added 2 commits 2026-06-26 16:07:05 -07:00

Carry live diagnostics in audio-alert popups ... 53e7d04e82

The audio_alert toasts showed only a static title and reason, so a user
who went inaudible couldn't tell which cause they'd hit. Add a structured
`details` list to the AudioAlert payload and fill it at each emit site with
the live values needed to triage: selected input device, software input
gain, silence duration, capture rate, mute state, and whether the capture
pipeline is running-but-silent vs exhausted. The encode-failure alert now
carries the actual encoder error (previously log-only). The frontend hook
becomes .tsx and renders the details as a key/value table under the reason.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Align @tauri-apps/api with the tauri crate (2.11) ... 01106ca0ef

`tauri build` aborts on a major/minor mismatch between the JS API package
and the Rust crate. The crate had advanced to 2.11 while @tauri-apps/api
was pinned at ^2 and resolved to 2.10; bump it to ^2.11 so release bundling
(MSI) runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

seb added 1 commit 2026-06-26 17:03:01 -07:00

Auto-recover a dead capture pipeline and report the cause ... f67d217b53

Audio alerts now name *why* the mic died: the fatal CPAL stream error
(e.g. WASAPI AUDCLNT_E_DEVICE_INVALIDATED) is captured from the stream
error callback into a shared slot and surfaced in the popup.

Replace the capture-liveness watchdog with a supervisor that detects a
dead pipeline two ways — frozen capture callbacks or an exhausted signal,
covering both an OS-torn-down stream and a crashed processing thread —
and rebuilds it automatically after a short grace (ignoring blips and
deliberate hot-swaps), with backoff and an attempt cap. It surfaces
"Reconnecting microphone…" with the cause and failure mode, falls back to
the system default if the selected device is gone, hot-swaps the fresh
signal into the running mic task, and shows a terminal "Microphone
disconnected" alert if recovery gives up. AudioManager gains
rebuild_input_signal; the mic task no longer raises the dead-pipeline
alert (the supervisor owns it) and keeps only the alive-but-silent case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

seb force-pushed issue/moq-port from f67d217b53 to 6743f04f69 2026-06-26 17:13:20 -07:00 Compare

seb force-pushed issue/moq-port from 6743f04f69 to 27cb284fe9 2026-06-26 17:35:40 -07:00 Compare

seb merged commit 13549846d2 into main 2026-06-29 10:12:49 -07:00

seb deleted branch issue/moq-port 2026-06-29 10:12:49 -07:00

seb referenced this pull request from a commit 2026-06-29 10:12:50 -07:00

MOQ port (#97)

Sign in to join this conversation.

Reviewers

No reviewers

puregarlic

Labels

Clear labels   

area

client

Concerns the Tauri app

  

area

server

Concerns the server component

  

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

ui

Involves updating the client UI

  

wontfix

This won't be fixed

No labels

area

client

area

server

bug

duplicate

enhancement

help wanted

invalid

question

ui

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

3 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

puregarlic/microclimate!97

No description provided.