AI customer service needs a refund test

Zoom's announcement is about more than putting a bot on the front door. Agent Architect can generate voice or digital agents from a prompt, while Agent Performance Suite lets teams simulate customer interactions before deployment and track resolution rate, containment, customer satisfaction, and cost per resolution after launch. Zoom also says its customer context layer reduces repeat explanations as people move across Zoom Virtual Agent, Contact Center, and AI Expert Assist.

Verint is moving in the same broad direction. Agent Factory gives contact centers a place to build and manage AI agents alongside human teams, with prompt management, model choice, data connections, governance, and human handoffs. Its separate Quality Intelligence release gives a cleaner example: if an agent promised a customer a $500 refund but entered $100 in the system, the tool is supposed to catch the mismatch the same day.

That refund example is the right level of specificity. AI support does not become useful because a team can launch agents faster. It becomes useful when the promise made in the conversation matches the action recorded in the business system.

Containment can mean two very different things. In the good version, the AI handles the whole issue and the customer leaves done. In the bad version, the company keeps the customer away from a human long enough that the complaint dies from exhaustion.

CNBC's April report on AI customer-service frustration cited Qualtrics data saying nearly one in five consumers who used AI for customer service saw no benefit. The same piece quotes Wake Forest's Shannon McKeen on the real irritation: customers are bothered by automation that traps them in a loop. That matches the everyday experience. People can tolerate automation that works. They hate automation that burns their patience and then asks them to re-explain the problem to a person.

Contentsquare's 2026 benchmark puts numbers around the gap. Its report says bot-only conversations resolved at 27%, while conversations with humans and bots together resolved at 50%. That does not mean every issue needs a person. It means the handoff is part of the product, not a failure path to hide.

A useful AI support test should look boring. Take 50 real cases from the last quarter: refunds, lost packages, billing mistakes, locked accounts, warranty questions, subscription cancellations, address changes, and policy denials. Strip private details. Run the AI against the same cases with the same knowledge base and tools it will use in production.

Then score the result after the business action, not after the chat answer. Did the refund post for the right amount? Was the replacement created? Was the cancellation completed? Did the denial cite the right policy? Did the agent preserve the conversation so the human could pick it up? Did the customer have to contact support again within seven days?

The scorecard I would want is short: completion rate, wrong approval, wrong denial, repeat contact, time to human, promised-versus-recorded mismatch, human review minutes, and customer effort. If those numbers move in opposite directions, say so. A cheaper ticket that creates a second ticket is not cheaper.

Small teams should start with bounded support work: order status, password resets, intake forms, simple policy questions, appointment changes, and gathering the context a human usually has to ask for twice. Let the AI handle the easy path. Make the escape hatch obvious before the customer gets angry.

For harder cases, the agent should do less and explain more. It can collect the order number, summarize the issue, check the policy, show what it cannot do, and route to the right person with a clean handoff note. That still saves time. More importantly, it avoids making the customer pay for the company's automation experiment.

Outcome-based pricing could help if the definition of outcome is strict. Resolved should mean the issue is actually done. Routed should mean the next person receives enough context to act without starting from zero. Anything softer will turn into a finance metric wearing a customer-experience costume.

Mina Torres reads this from the customer's side. Her concern is simple: if the bot asks five questions and the human asks the same five again, the company saved time by spending the customer's time. The first visible improvement should be fewer repeat explanations.

Sable Quinn sees the positioning problem. People will not repeat 'Agent Factory' to a friend. They will repeat 'the bot promised me $500 and I got $100.' If the new tools catch that before the customer notices, there is a real story. If they only produce cleaner dashboards, the story gets ugly fast.

I am closer to Mina on rollout and closer to Sable on proof. Do not announce the agent as smarter. Show the refund, the handoff, the denial reason, and the repeat-contact rate. Support is one of the few AI categories where the customer can tell immediately whether the metric is fake.