Start with the conversation, not the stack

The first decision in a voice AI MVP is not the model or the speech provider. First you need to understand the conversation the product must survive. A caller arrives with intent, interrupts, pauses, speaks through noise, changes direction, asks for repetition, and sometimes leaves the happy path entirely.

If that path is not described, the technical stack can become a polished showcase without durable behavior. A good MVP starts with one scenario, a clear boundary of responsibility, and an honest list of moments where the agent should hand off to a person or another process.

Latency is UX, not just a metric

In a text interface, delay often feels like waiting. In voice, it feels like a broken conversation. That is why the latency budget has to be split across the whole chain: VAD, transport, recognition, reasoning, tool calls, synthesis, and playback.

The useful early work is not chasing perfect milliseconds. It is understanding where delay is inevitable, where conversation design can hide it, and where the architecture itself needs to change.

An MVP should produce artifacts

After the first prototype, the team should have more than call recordings. It should have artifacts: a pipeline map, fallback scenarios, latency table, quality criteria, integration plan, and a list of remaining risks.

That kind of MVP helps the team make decisions. It shows what to automate first, where humans still matter, what data the agent needs, and which compromises are acceptable for the business.

Illustrative scenario: booking an appointment

Imagine an agent taking a call and offering available times. This is a design example, not a deployed case study: the agent confirms a booking only after the calendar succeeds and hands off when the operation cannot be completed.

One-call review template

  • Outcome: define the required details, the exact point at which a booking is confirmed, and the response when no slot is available.
  • Latency: for each test call, record speech end, transcript readiness, model response, calendar completion, and first audio. Compare median and 95th-percentile time per stage on the same call set.
  • Failures: test silence and noise, interruption during playback, calendar timeout, and a repeated request. Give each case a next step without inventing a confirmation.
  • Acceptance: the recording and event log should show whether exactly one booking was created, what the caller heard, and when a person was needed.