See exactly why a call passed, or failed.

Score every call across five dimensions, on your own infrastructure.

$ uvx hotato start --demo

Everything runs on your machine.

See what one run checks ↓

hotato · scorecard · support-bot

conversation-test.v1 · agent support-bot · call 8f2a

Outcomerefund posted to the ledgerdeterministicpass

Policyrequired disclosure readdeterministicpass

Conversationtalked over the caller 1.32sdeterministicfail

Speechlatency within budgetdeterministicpass

Reliability20 runs, all helddeterministicpass^k

Model judgetone, sent to reviewadvisoryinconclusive

Five dimensions, each scored separately. The deterministic verdict stands on its own.

3,544tests in CI
17assertion kinds
5dimensions
0runtime deps

Five questions, checked five ways.

The transcript reads clean, yet the refund never fired. The scorecard says why.

Outcome did the task succeed
Policy did it follow the rules
Conversation did it listen and take turns
Speech transcript, latency, audio
Reliability does it hold across repeats

The written Hotato report for one call: per-dimension results, the turn-taking timeline, and the evidence behind each verdict. — The report for one call: per-dimension results, the turn-taking timeline, and the evidence behind each verdict.

Explore each dimension →

One file. Every check.

A conversation test is one YAML file: 17 deterministic assertion kinds, plus one advisory rubric lane.

conversation-test.v1.yaml → hotato test run

agent: support-bot
assertions:
  # 17 deterministic kinds, no model
  - kind: outcome      # refund posted to the ledger
  - kind: policy       # required disclosure present
  - kind: tool_call    # issue_refund ran, from the trace
  - kind: pii          # card number never spoken back
  - kind: latency      # reply gap under budget
  rubric:            # advisory lane, separate
    - kind: judge_rubric # deterministic:false

$ hotato test run conversation-test.yaml
  outcome        pass
  policy         pass
  conversation   fail   talked over caller 1.32s
  speech         pass
  reliability    pass^k 20/20 held
  model judge    advisory · sent to review
exit 1

The 17 kinds & the outcome authorities →

Confirm it once. CI replays every commit.

A confirmed failure becomes a portable contract; CI re-scores it and exits 0 or 1.

$ hotato contract verify contracts/ --junit contracts.xml

contract verify · CI

contract verify

$ exit 1 $ exit 0 the same contract, before and after the agent yields

A self-hosted control plane.

Read-only views over your registry that recommend what to look at before you ship.

Runs in your VPC
bearer-token auth
zero egress

Release-readiness view: current release, failures by dimension, required suites and what changed since the last release. — **Release readiness**: does this release clear its required suites, and what changed.

See the workspace →

One run covers the rest.

The same command simulates at scale, times every turn, tiers your evidence, and stays offline.

Simulate & pass^k

Deterministic callers replay a scenario across seeds; pass^k reports whether it held on every run.

Simulation →

Deterministic timing

Millisecond turn-taking on two channels, no model in the path. It times when each side spoke.

Timing →

Evidence tiers

A machine-readable vector; recompute-from-audio and an Ed25519 signature raise the tier.

Trust & evidence →

Zero egress

Offline by default; a not-scorable input is refused with a plain reason, never a guess.

Self-host →

Integrations

Capture from Vapi, Twilio, Retell, LiveKit, and Pipecat, without changing your stack.

Integrations →

MCP & agents

Drive every run from an agent over MCP, with machine-readable output and exit codes.

Docs →

The moments no single threshold catches.

Hotato ranks the candidate interruption moments; you label each one.

Closing those moments takes engagement control at the model layer. That is what Attention Labs builds.

your recordingcaptured
→
hotatoper-dimension scorecard
→
candidate momentsranked, you label
→
Attention Labsconsent + focus gated

Talk to us →

Terminal output of hotato investigate: capture origin operator-asserted, eligibility, and three ranked candidate interruption moments each awaiting a yield-or-hold label. — Capture origin stays **operator-asserted**; the candidate moments are ranked for you to label.

The boundaries.

What Hotato measures, and what it deliberately leaves to you.

What's the overall accuracy score?

None, by design. Five dimensions are reported separately, never averaged into one number. The deterministic checks re-run identically, and that reproducibility gates CI.

Does it read intent, emotion, or who is speaking?

No. The timing layer measures energy on two channels and flags candidate moments for you to label; energy is not intent. It identifies no speakers: each channel already belongs to one party. A transcript is opt-in context for phrase and PII checks, never itself scored.

Can I run it on one recording?

Yes. uvx hotato start --demo runs the full flow on bundled calls, no account or network. Point it at one of your own for a per-dimension scorecard.

What does it cost?

Free and MIT-licensed, with zero runtime dependencies and no phone-home. Every dimension, the workspace, the judge, and the simulators are included.

Which voice stacks?

Capture from Vapi, Twilio, Retell, LiveKit, and Pipecat. Placing a live billed call is Vapi and Twilio only, gated. Integrations →

Does my audio leave my machine?

Not on the default path. Scoring, assertions, a local judge, and the workspace run offline. A recording moves only when you pull it or opt in. Security →

Know what your agent did on every call.

Bundled recordings, a per-dimension scorecard, and a CI-ready contract, in one command.

$ uvx hotato start --demo

One recording is enough to start. Get started → · self-host →