When to build a voice interface (and when a form wins)

Every product team, at some point in the last few years, has had The Voice Meeting. Someone returns from a conference with a demo of a model that hears, speaks and reasons in real time, and the meeting ends with a ticket that reads "explore voice". Three months later the exploration is a prototype that sounds miraculous in the boardroom and gets used twice by actual customers — once to try it, once to confirm it doesn't understand them.

Voice is not a feature you add. It's a modality you earn. The teams that ship voice well start from a rude question: what can a user say faster, safer or more comfortably than they can tap? When the honest answer is "nothing", the form wins, and shipping voice anyway is how you burn a quarter. When the answer is real — hands busy, eyes elsewhere, literacy barriers, a paragraph of dictation that would take forty taps — voice becomes one of the best interfaces ever built. This is the framework we use on engagements in our AI practice to tell those two situations apart before anyone books the voice talent.

The four conditions where voice genuinely wins

Speech has a brutal economics: speaking is fast (around 150 words a minute), listening is slow (you cannot skim audio), and correcting speech errors costs more than never making them. Voice wins only when that trade flips in the user's favour. We've found four conditions that reliably flip it:

Hands or eyes are legitimately busy. Not "could be" busy — a warehouse picker mid-scan, a nurse mid-procedure, a driver, a home cook with flour on their wrists. The test: is the alternative to voice not a keyboard but stopping a task? On a logistics client we shadowed, pickers were pausing 6–11 seconds per item to key confirmations into a rugged terminal. Voice confirmations cut the pause to under two. That's a modality earning its keep.

The input is long-form and unstructured. Dictating a paragraph of clinical notes, describing a fault in your own words, rambling through symptoms. Typing that on glass is misery; speaking it is natural. This is why intake and documentation flows — the kind we designed for Pylon Health's telehealth experience — are the strongest voice candidates in consumer software.

Accessibility is the primary driver. For users with motor impairments, low vision or low literacy, voice can be the difference between using the product and not. Note the ordering: this is a reason to build voice properly, with the same rigour as the visual UI — not a marketing garnish. Our accessibility audit process treats voice paths as first-class journeys for exactly this reason.

The context is ambient and shared. Meeting rooms, kitchens, cars. Spaces where a screen is socially or physically wrong, and where more than one person is party to the interaction.

Notice what's absent: "it's delightful", "it's futuristic", "our competitor has one". Those are how prototypes get funded and features get abandoned.

Where the form still wins

Be honest with this list, because it's long:

Precise, structured data. Dates, times, quantities, addresses. "Friday" was a lie told by every calendar app of the 2010s. A date picker is deterministic; speech recognition of "the thirteenth or was it the thirtieth" is a coin toss with a lawsuit attached.
Skimmable output. Comparing six flights, scanning a bill, reviewing a contract. Listening is linear; eyes are parallel. If the job is comparison, voice is cruelty.
Public or quiet environments. Nobody dictates their medical history on the 7:42 to Central. This kills a surprising number of proposed consumer voice features the moment you ask where users actually are.
Correction-heavy tasks. Anything where a wrong value is expensive and error rates aren't near zero — payment amounts, names, medication dosages. Confirmation loops erase all the speed voice promised, and then some.

The strongest products mix modalities: voice for capture, screen for verification. Speak the appointment; see the summary card; tap to confirm. The point of multimodal design is that no modality has to carry weight it can't bear.

Latency and turn-taking: the feel is the feature

Humans read conversational silence fast. Under roughly 300 milliseconds, a response feels instant. Between 300 and 800 it feels considered. Beyond about a second, users start wondering whether the thing heard them; beyond two, they repeat themselves, and two overlapping audio streams collapse the whole exchange. In our testing, perceived responsiveness mattered more to satisfaction scores than raw transcription accuracy — people forgive a misheard word far sooner than a dead pause.

Design for it, don't hope for it:

Acknowledge before you answer. A subtle listening state — a responding waveform, a soft "mm" cue in voice-only contexts — buys you the thinking time the model needs. This is streaming UX applied to the ear: make waiting feel like thought, not failure.
Endpoint with intent. Silence-based endpointing chops eager speakers mid-thought and waits an eternity on hesitant ones. Tune silence thresholds per use case (a form field needs faster endpointing than open dictation), and let the user press-to-talk as an escape hatch.
Support barge-in. Real conversation is interruptible. If a user says "no, not that one" while the assistant is mid-sentence, the assistant must stop now, not finish its paragraph like a politician on breakfast television. Full-duplex audio with interruption handling is the difference between talking with software and submitting to it.
Recover visibly. "Let me say that differently" is a feature, not an apology. Every correction path should be shorter than re-doing the task by hand, or users will simply do it by hand.

Transcription honesty: show the confidence

A voice interface makes two kinds of mistakes: it mishears, or it hears perfectly and misunderstands. Both are survivable; silent versions of both are not. The rule we ship: nothing the system heard is invisible. Every voice input produces a visible transcript before anything consequential happens. Where recognition confidence is low, the interface says so — highlighting the uncertain word beats confidently rendering the wrong one, and it's cheaper than the confirmation dialogues you'd need otherwise.

Calibrate confirmation to consequence:

Low stakes, reversible (playing a track, setting a timer): act immediately, offer undo.
Medium stakes (sending a message, booking a slot): transcript + one explicit confirm.
High stakes (money, medicine, anything legal): voice proposes, a deterministic UI disposes. The Beacon Health triage assistant follows this to the letter — speech helps people describe symptoms in their own words, but every clinical hand-off happens on a rendered, reviewable summary. Knowing when to stop talking was the whole design.

This is the pattern behind trustworthy agentic systems generally — the same discipline as designing for delegation: the further the action is from reversible, the more the interface slows down and shows its working.

The privacy posture voice demands

A microphone changes the stakes of your data story. Users know what a form captures; a mic feels like it's always capturing, whether or not it is. Design accordingly.

Keep wake and recording states unmissable — not a 4-pixel dot, an unambiguous state a person can clock from across the room. Prefer on-device or streaming recognition with a clearly stated retention policy; "audio is processed and discarded" is a product feature worth a settings screen, not a clause on page nine of the privacy policy. Give users a way to review and delete their transcripts — treating voice history like search history is table stakes. And never let a voice feature in a shared space act on one person's instruction without making the action visible to everyone present. Ambient computing without ambient accountability is how press coverage happens.

Get the posture right and voice earns something keyboards can't: trust built through a sense of being heard — literally. Get it wrong and no accuracy metric will save the feature, because users will never turn it on.

Key takeaways

Voice wins where speaking is faster or safer than typing: busy hands, long dictation, accessibility needs, ambient contexts. Delight is not a condition.
Forms still own structured data, comparison, quiet places and correction-heavy work. Mix modalities — speak capture, see verification.
Under a second feels alive; past two it feels broken. Acknowledge quickly, endpoint by intent, and support barge-in.
Show transcripts always, signal low confidence, and match confirmation friction to consequence.
A visible, deletable, honestly-retained audio trail is the price of admission for anything with a microphone.

FAQ

How accurate does speech recognition need to be before we ship? Accuracy thresholds depend on consequence, not benchmarks. For dictation where users correct inline, word-error rates in the mid-single digits are tolerable. For anything that triggers actions, what matters is the error detection loop: can the user see and fix a mishearing cheaper than doing the task manually? Test with your real vocabulary — names, jargon, accents of your actual user base — because published benchmark numbers describe somebody else's customers.

Should our voice feature work fully hands-free, or is press-to-talk acceptable? Press-to-talk is an excellent first version: it solves endpointing, gives users explicit control, and reads as intentional rather than creepy. Go fully hands-free only when the context demands it (driving, surgery, kitchens) — and treat wake-word false positives as a privacy bug, not an accuracy statistic.

Do we need a custom voice, or is the platform default fine? Start with the platform default and spend your budget on turn-taking and error recovery — that's where experiences live or die. A custom voice matters when the assistant is the brand's front door (a bank's service line, a media brand's companion). Even then, voice casting is one workstream inside a sound design budget, not a substitute for conversation design.

How do we prototype a voice flow before committing to the stack? Wizard-of-Oz it. One teammate plays the system, reading scripted responses with deliberate latency. You will learn more about turn-taking, confirmation load and phrasing in three afternoons of this than in a month of SDK evaluation — and the script you end up with becomes your conversation spec.