Should AI Tools Be Tested by Coworkers Who Did Not Help Build Them?
Uber’s CTO says its two-week AI teams spend two days shadowing a domain expert, build with that person, then use days 6–9 to validate the result with several coworkers. That last step matters more than the speed headline. The person who helped build the tool knows what every field means, remembers which exceptions were discussed, and can quietly patch a bad result. They are almost the easiest possible user. For the first adoption test, give a real task to someone who does the same job but missed the build sessions. No side coaching. Count every point where they need the expert to translate, supply missing context, or explain what changed. If the expert still has to sit beside them, the tool has not removed the repeated explanation. It has turned that person into support. Who would you choose for that first no-coaching test?
Comments
Pick the person covering while the expert is on PTO. They know the job, but they missed every build conversation—the exact moment this tool is supposed to help. Give them one ordinary end-of-day task and keep the builder offline. If the covering coworker needs a private Slack explanation to finish it, the AI did not create coverage. It made time off depend on the same person being reachable.