Hotato docs

Open-source, self-hosted conversation QA for voice agents. Verify task outcomes and policy, catch voice failures, and keep every result reproducible on your own infrastructure — five dimensions, never one blended score, with the evidence behind every result.

What it is

MIT, zero-install CLI plus an MCP server and a self-hosted workspace. A conversation test checks a call across five dimensions — Outcome, Policy, Conversation, Speech, Reliability — using seventeen deterministic assertion kinds and a separate advisory model judge. Pass, fail, or inconclusive; a non-zero exit gates CI. Never one blended score.

Deterministic checks read the trace, the system of record, or the audio and run no model. Turn-taking timing measures energy over time, not intent — no speaker ID, diarization, transcription scoring, or emotion detection. How it works covers the method.

Five ways to run it

The demo catches a failure with zero setup; a conversation-test scores your own call across the five dimensions. Verdicts come only from scoring your own call.

# 1) zero-setup first run: sweep the bundled calls, mint + verify a failure contract
uvx hotato start --demo

# 2) the atomic unit: author a conversation-test.yaml, then score a call across five dimensions
uvx hotato scenario init refund-flow --out refund.conversation-test.yaml
uvx hotato test run refund.conversation-test.yaml --agent support-v3 --transcript call.json --trace trace.jsonl

# 3) score your OWN call: capture auto-pulls the dual-channel recording, then scores it offline
uvx hotato capture --stack vapi --call-id <id>

# 4) the 5-minute path: score, render the visual report, open it, one command
uvx hotato doctor --stereo call.wav

# 5) connect once, then rank candidate moments across recent calls
uvx hotato connect vapi
uvx hotato sweep

Where to go next