Home/Docs/Research export
Reports

Research export

A pass/fail exit code won't defend a threshold change to a skeptical teammate. hotato export scores a recording (or the battery) like hotato run, then writes the evidence to three flat files a notebook or stats package can open directly. Stdlib only, offline.

score the recording events.csvone row / scored event frames.csvone row / VAD frame envelope.jsonone entry / run
One scoring pass, three files: the verdicts, the frame-level evidence behind them, and the same machine envelope hotato run writes.
bash
uvx hotato export --stereo call.wav --out research/
uvx hotato export --suite barge-in --out research/

The three files

FileOne row perWhat it carries
events.csvscored eventevery measured signal plus the verdict: did_yield, seconds_to_yield, talk_over_sec, onset, latency signals, fix class
frames.csvVAD framethe evidence behind every number: t_sec, per-channel dBFS, active flags, thresholds, and noise floors
envelope.jsonrunthe standard machine envelope, unchanged

Flags

export takes the same single-recording and battery inputs as hotato run, always against the energy VAD reference: there is no --mono/diarization path here, so a two-channel recording is required.

  • One recording: --stereo file.wav (caller and agent on separate channels of one file; --caller-channel and --agent-channel default to 0 and 1) or --caller plus --agent as two mono files. --onset overrides caller-onset auto-detection; --expect yield|hold sets what the agent should do.
  • A battery: --suite NAME (default barge-in) against --scenarios and --audio directories, defaulting to the bundled fixtures, the same directories tiered suites and hotato report take.
  • --stack {generic,vapi,twilio,livekit,pipecat,retell} records which voice stack the recording came from. It only labels the fix_class column; no measurement changes.
  • Regression gates: --max-talk-over and --max-time-to-yield fail the run the same way they do in hotato run; --no-fail always exits 0 so a research export never breaks a CI job that only wants the files.

Self-describing columns

Column meanings are documented in comment lines at the top of each CSV, so the files still explain themselves when they land in a notebook or a stats package months later.

events.csv
# hotato events export. One row per scored event.
# Empty cell = not derivable from this recording, never a stand-in value.
# Columns:
#   event_id              stable id of the event (file basename or scenario id)
#   scenario_id           battery scenario id (empty for a single recording)
#   title                 human label of the event
#   ...
Empty means empty

An empty cell marks the case where the recording didn't yield that value: never a zero, never a stand-in. Frame rows are re-derivable by hand: active is dbfs >= threshold carried by the hangover, and both constants sit in the same row.

The latency SLA gate

export pools response gap (dead air before the agent speaks) across every exported event and prints mean/median/p90/p95, the same pooled definition hotato team reports. --max-response-gap SECONDS turns the pooled p95 into a bound and fails the run the moment it is exceeded.

terminal
$ hotato export --suite barge-in --out research/ --max-response-gap 0.1
wrote 8 event rows to research/events.csv, 5150 frame rows to research/frames.csv, and the envelope to research/envelope.json
response gap: mean 0.34s median 0.34s p90 0.34s p95 0.34s (n=1)
latency SLA: p95 response gap 0.34s vs bound 0.10s (fail)

Exit 0: exported, every scorable event passed. Exit 1: a scorable event failed, or the latency SLA was breached. Exit 2: usage error or unusable input. --no-fail always exits 0, for a CI job that only wants the files.

Works with the whole surface

The same scenario and audio directories work here too: export a tiered suite, a captured call (capture guides), or your own labelled set with one flag change. A frozen recording re-exported later reproduces the same three files byte for byte: a stored-evidence check, useful as a regression fixture, never proof your live agent still behaves the same way today.