Capture from LiveKit

LiveKit egress: caller and agent as separate tracks. setup prints the scaffold; hand both to Hotato to score offline.

The recording you capture feeds a five-dimension conversation-test: separated caller and agent channels are what the Conversation (turn-taking) and Speech (timing) dimensions read, and the same file drops into hotato test run beside a transcript and trace for the Outcome and Policy checks.

Stack	Automatic pull	Separate channels	Contract-grade default	Network use
Vapi	Yes: `capture` fetches by call id	Yes, out of the box	Yes	One call: fetch the recording
Twilio	Yes: `capture` fetches by recording SID	No by default: must request `RecordingChannels=dual` at creation	No, opt-in	One call: fetch the recording
Retell	Yes: `capture` fetches by call id	Yes, once call recording is turned on	Opt-in: enable recording first	One call: fetch the recording
LiveKit	No: egress runs in your own infra	Yes, if egress records both tracks	Opt-in: configure egress for two tracks	None from Hotato
Pipecat	No: the `setup` recorder runs in your pipeline	Yes, the scaffold writes caller ch0 / agent ch1	Yes, once the recorder is added	None: local recorder, no network call

Automatic pull: does hotato capture fetch it for you. Separate channels: is caller/agent separation what you get. Contract-grade default: is that separation on without extra setup. Network use: what capture calls out to, not your infrastructure.

Egress records the call; Hotato scores it afterward, once that recording clears LiveKit’s export pipeline; that gap is what this guide walks through. The adapter instead writes the two-channel WAV inside your own script, so scoring starts the moment capture ends, no export step between. Same input contract either way: caller and agent on two tracks.

1. Print the egress scaffold

setup prints a config for separate caller/agent tracks. Room-composite egress mixes both parties into one track; use track egress, since the scorer needs separated caller and agent tracks.

Track egress writes caller and agent to separate channels, so the scorer can tell who cut in on whom. Room-composite egress collapses both into one channel: ownership is lost. Use the two-track path.

bash

uvx hotato setup --stack livekit

2. Score the two tracks

Hotato scores both tracks as two channels, offline. They need to start at the same instant: nothing here re-syncs a caller and agent captured on separate clocks. --expect yield/hold = interruption vs. backchannel.

bash

uvx hotato capture --stack livekit --caller caller.wav --agent agent.wav

# no files yet? prove the capture→score loop on a bundled call
uvx hotato capture --stack livekit --demo

Two-channel WAV already? Score directly with uvx hotato run --stereo call.wav --stack livekit --expect yield (caller on channel 0, agent on channel 1).

Or capture live, without egress

adapters/livekit_capture.py is a wiring template that skips egress: it plays a caller stimulus into your own AgentSession and records the agent’s output track directly, writing the same two-channel WAV capture scores. Three marked ADJUST points connect it to your session, publish the stimulus, and drain the agent’s track; unwired, it raises rather than guessing. Needs the optional stack: pip install 'hotato[livekit]'.

Source to wire: livekit_capture.py.

Scoring more than one call?

LiveKit has no vendor recording API to poll, so pull and sweep have nothing to fetch here. Point hotato analyze at the folder your egress or adapter writes to instead: same ranking sweep produces, over every WAV inside.

bash

uvx hotato analyze ./recordings

The dual-channel gate

Separated caller and agent audio is what makes overlap attributable; speech activity comes from exposed VAD thresholds, frame-inspectable. The scorer takes one two-channel WAV, or two aligned mono WAVs. LiveKit has no fetch step for --allow-mono to gate: a room-composite (single mixed-down) file is rejected outright, with or without the flag. Reconfigure two-track egress above, or separate an existing mono recording with hotato run --mono file.wav --diarize (opt-in, quality-gated; see how it works).