Voice is widely expected to become one of the next major computing interfaces. Investors have poured billions of dollars into voice AI startups, backing companies that build foundation models, automate enterprise customer service, take meeting notes, and power dictation. New models arrive almost every week, each promising more human-like speech and smoother conversation. Yet senior executives working on the technology say the field has not reached its defining consumer breakthrough — the kind of moment that made text-based chatbots feel suddenly useful to a mass audience.

At an industry conference, PolyAI CTO Shawn Wen argued that full-duplex models, which can listen and speak at the same time, mark an important technical milestone but not the finish line. The next challenge is making reasoning fast enough that models can retrieve answers quickly and keep a conversation natural. In customer service, he said, AI agents also need to avoid sounding robotic and must give callers confidence that their problems can actually be solved.

Key Facts

• Voice AI has strong investment momentum but has not yet had a ChatGPT-style breakthrough, according to executives.
• PolyAI CTO Shawn Wen says full-duplex models are a milestone, but fast reasoning and natural conversation remain challenges.
• Wen argues customer service agents must sound less robotic and build caller confidence over the first few turns.
• Otter CMO Alex Gay says speaker identification, intent capture and linking transcripts to organizational knowledge are key for automation.
• Otter is exploring digital twins that could represent people in meetings, requiring emotionally expressive voice output.
• Transcription accuracy remains critical because errors cascade into flawed summaries and actions, eroding trust.
• Executives stress that tools should disclose recording and AI use to customers and meeting participants.

Full-duplex models are a milestone, not the finish line

Full-duplex voice models represent a meaningful advance over earlier systems that had to wait for a user to finish speaking before generating a response. The ability to listen and speak at the same time is closer to how humans conduct conversations, with interruptions, backchannel cues, and overlapping speech. But Wen said the industry has reached the milestone of developing full-duplex models, and the next challenge is to make reasoning very fast so that models can fetch answers quickly and the conversation feels natural.

That latency issue is central to the gap between a impressive demo and a dependable product. In a text chat, a short pause while a model thinks can feel acceptable. On a phone call, even a second of silence can feel like a breakdown. Callers may repeat themselves, talk over the agent, or lose confidence. If the agent needs to query multiple systems, verify identity, check account details, and apply a policy, the reasoning and retrieval steps must happen inside a tight conversational window. The model must also know when to ask a clarifying question, when to summarize, and when to hand the call to a human.

Wen said AI agents in customer service should not sound robotic and should give callers enough confidence that they can solve problems. He described a gradual trust-building process. Once the voice is good enough, and the customer is willing to engage with the agent for the first two or three turns, they start to build confidence. Over time, they may feel they probably do not have to talk to a human if the agent can solve the problem.

That first handful of turns is where voice AI either earns trust or loses it. A caller who hears a natural greeting may stay on the line, but if the agent misunderstands an account number, misreads an address, or fails to grasp the reason for the call, the illusion of intelligence collapses. Enterprises therefore judge voice agents not only on voice quality but on containment rates, resolution times, escalation rates, and customer satisfaction. A system that sounds human but cannot complete tasks can damage a brand more than a plainly automated menu.

Customer service needs confidence, not just conversation

For enterprises, deploying voice AI is an operational decision as much as a technical one. Customer service calls often involve authentication, billing, scheduling, returns, troubleshooting, and policy exceptions. Each step may require access to different systems and different levels of permission. The agent must understand intent, extract the right entities, retrieve accurate information, and act within business rules. It also must recognize when a problem is too sensitive, too complex, or too ambiguous for automation.

Wen’s emphasis on confidence reflects a broader truth about adoption. Customers do not judge a voice agent solely by whether it can produce fluent sentences. They judge it by whether it understands the problem, remembers context, and reaches a useful outcome. That requires fast reasoning, reliable tool use, and a clear path to human help. When those pieces work together, voice AI can move from a novelty to a practical layer in customer service. When they do not, callers quickly ask for a representative.

The market has responded with a wave of investment. Funding has gone to model makers building speech-to-speech systems, startups focused on enterprise customer service, meeting note-takers, and AI-powered dictation tools. Each category faces different barriers. Customer service demands compliance, integration, and low latency. Meeting tools demand speaker identification, accurate transcription, summarization, and secure handling of sensitive conversations. Dictation tools demand high accuracy across accents, jargon, and noisy environments. The common thread is that voice is only the surface. The real product is understanding.

Meeting AI faces accuracy and emotional nuance

Otter CMO Alex Gay said speaker identification, intent capture, and typing that up with organizational knowledge is a key step for enabling automation. In a meeting, a transcript without speaker attribution can be confusing. A summary without intent capture can miss decisions. Notes that are not linked to the organization’s knowledge base may fail to surface relevant context. The value of meeting AI comes from turning raw conversation into structured, searchable, actionable information.

Otter is also working on digital twins that might represent people in meetings. For that technology, Gay said it is paramount that the output voice gives the same emotive expressions of talking to a human in a meeting. The goal is not merely an avatar that can recite facts. It is an avatar that can participate in debate, strategic discussion, and the relationship-building that makes a meeting useful. Gay noted that the best conversations are those where people can debate, discuss strategy, and feel that a relationship underpins the exchange. If an avatar cannot do that, it is just a question-and-answer chatbot.

That comparison highlights how far voice AI must travel. A Q and A chatbot can answer a direct question. A meeting participant must listen, track multiple speakers, understand disagreement, infer priorities, and respond with appropriate tone. It must know when to push back, when to agree, and when to stay silent. Emotional expression is part of that, but so is social awareness. The technology raises new questions about identity and consent. If a digital twin represents a person, colleagues need to know whether they are talking to the person, a model, or a hybrid. The voice may be synthetic, but the trust issues are real.

Transcription errors can cascade

While voice AI models have improved, AI assistants often do not understand users, or a meeting note-taker shows the wrong transcript or a summary. Wen thinks automatic speech recognition models often miss important keywords, and that creates an issue in capturing the whole context. A missed product name, a misheard date, or a dropped negation can change the meaning of a sentence. In customer service, a single error can send a call down the wrong path. In a meeting, a single error can turn an action item into a misunderstanding.

Gay agreed with this, adding that the company keeps working on improving transcription. He also said language is one area where voice models need to improve. For Otter, transcription was never the endpoint. It was the layer that could start to drive productivity gains. But if the original transcription did not have the accuracy needed, all the follow-up actions become flawed. The minute the system starts to take action, that is wrong. Users lose trust in the platform. That is why it is critical to continue improving the ASR model: the downstream impacts are significant.

The cascade is easy to trace. A transcript feeds a summary. A summary feeds action items. Action items feed tasks, calendars, and customer records. If the transcript misidentifies a speaker or misses a keyword, the summary may attribute a decision to the wrong person. The action item may be assigned incorrectly. The customer record may contain the wrong commitment. By the time a human notices, the error has traveled through several systems. Voice AI vendors therefore cannot treat transcription as a commodity layer. It is the foundation for automation.

Transparency becomes a product requirement

With new voice tools, there is also a question of transparency. Tools should declare to customers that they are being recorded or talking to AI. Otter said it wants to instill trust in people who are in a meeting, so even for a meeting where the bot is not present, it wants to try methods like notifying everyone in the chat that the meeting is being recorded. PolyAI’s Wen also said it is important to establish that people are talking to an AI in enterprise calls.

Disclosure is both an ethical obligation and a practical adoption strategy. Customers who feel tricked by a synthetic voice may react badly when they discover the truth. Meeting participants who are recorded without notice may lose trust in the platform and the organization. Enterprises operate under a patchwork of rules covering call recording, consent, data retention, and AI disclosure. A voice AI product that ignores those rules may create legal and reputational risk. A product that builds disclosure into the workflow can reduce friction and make the technology easier to deploy.

The executives’ comments suggest that voice AI is moving from demo to deployment, but the bar is high. Full-duplex models are necessary, not sufficient. The next phase will be defined by fast reasoning, accurate transcription, emotional nuance, and transparent design. Voice interfaces have been predicted for decades, from interactive voice response systems to smart speakers and virtual assistants. Each wave promised natural conversation and each wave exposed the difficulty of understanding real human speech. Large language models have raised expectations again, but they have not solved the last mile of trust.

For enterprise voice AI, the bar is not just sounding human. It is being understood, trusted and transparent enough that people are willing to keep talking.

Source:TechCrunch News