How should AI voice features in Gmail and Docs be measured?
Voice is useful when typing is the bottleneck, but a clean transcript is not the finish line. For Gmail and Docs, compare a week of similar messages or drafts made by voice and keyboard: time to a ready-to-send version, edits to names, dates, and commitments, recipient follow-up questions, and anything corrected after sending. Fewer keystrokes do not help if somebody still has to translate a muddled draft before it can leave the building. What would you measure first?
Comments
One number I’d keep: messages the sender has to correct after the recipient reacts. Dictation can turn ‘I can probably do Friday’ into a firm yes. Draft time looks great until the repair needs an apology. Count that separately from typos; it catches when the tool changed the social meaning, not just the words.
I’d put one mundane test beside the numbers: dictate a reschedule while you’re leaving the grocery store, then reopen it an hour later. Can you see the exact day and time, and tell it is still a draft, without asking the assistant to explain itself? Voice earns its keep when a two-sentence message does not create a second task later.
Also keep the repair where the mistake is. If the voice draft says ‘Friday’ after you meant Thursday, tapping that date should let you hear the short captured phrase and change just that field—not dump you back into a full transcript or a blank prompt. On a phone in a parking lot, people will fix one wrong commitment; they will abandon a tool that makes them edit a whole email to prove it heard them wrong.
Add a source conflict to that reopen: one allowed email says Thursday, the calendar says Friday. Google says Docs Live may pull from Gmail, Drive, Chat, and the web with permission; its announcement does not say how a draft exposes the source it chose. If the user cannot see that after an hour, the saved draft is only a more legible guess.
If the message has already turned Thursday into Friday, the draft should say it chose between sources—not make the sender reverse-engineer the mistake. Saving keystrokes is nice. Quietly picking the commitment and borrowing your name for it is not.
That is where a small team can lose the time it thought it saved. If a tentative answer goes out as a promise, the person covering the customer should see the original draft, the correction, and whether anyone replied before they inherit the thread. Otherwise they get the apology with none of the context.