How live UAT with insurance advisers reshaped an AI sales coach
At Mindreader, live UAT with insurance advisers showed where technically valid AI output became operationally useless, and the evidence became a narrower product and iteration plan.
The customer context
Mindreader was an AI-assisted sales platform for people selling high-consideration products. It combined a client communication profile with practical tools such as meeting role-play, sales-script generation, and contextual sales advice.
Insurance advisers were a natural early audience: they managed many client relationships, prepared for sensitive conversations, and often worked between WhatsApp, spreadsheets, internal systems, and meetings on the move.
The product question sounded simple: could AI help an adviser understand a client and decide what to say next?
The assumption that broke
Our initial assumption was that accurate client insight plus an open-ended AI coach would be self-evidently useful. Once an adviser could see a communication profile, the role-play and script tools would help them act on it.
Observed usage showed a more complicated reality:
- Some users did not understand what a probability bar meant or what they should do after reading the insight.
- The journey required too much switching between a phone, work email, WhatsApp images, and the desktop product.
- Generated dialogue could be technically aligned with a profile and still sound too formal, too long, or unlike anything the adviser would say.
- Experienced advisers often felt they already knew how to handle the conversation. Newer advisers had a clearer need for practice and coaching.
The bottleneck was not generating more insight. It was helping the right adviser turn that insight into a credible next action inside the way they already worked.
How I investigated it
I ran and documented live user-acceptance sessions with four insurance advisers. Instead of asking whether they liked the idea, I watched where they clicked, what they tried to enter, when they hesitated, and how they compared the generated output with real client conversations.
The sessions covered the full path: onboarding, creating a client profile, interpreting the result, opening the AI tools, practising a conversation, and generating a message or script.
This surfaced evidence that a feature survey would have missed:
Workflow friction appeared before model quality
One participant tried to move a client image from WhatsApp but stopped because downloading and uploading it was too cumbersome. Another switched devices to complete sign-up because her personal computer could not access her work email.A correct output could still leave the user stranded
After receiving the analysis, users repeatedly asked some version of “what is next?” The product exposed insight, role-play, and scripts as separate tools instead of one guided sequence.The same feature had different value by experience level
A team lead with four years in the industry said he did not need the chatbots for his own conversations, but saw a stronger use case for coaching newer consultants.Realism and learning pulled in different directions
A role-play bot that exaggerated a communication style could teach the underlying pattern, but it could also feel artificial. The product needed enough realism to be credible and enough contrast to be educational.
Translating the evidence into a system
The discovery narrowed both the user and the job to be done. Rather than treating every adviser as someone who needed an AI answer, the stronger wedge was guided practice for less-experienced advisers, with team leaders able to use the same system as a coaching aid.
The boundary became clearer:
| AI should handle | The adviser should own |
|---|---|
| Simulating a client communication style | Choosing a real and relevant sales scenario |
| Generating alternative openings and responses | Judging whether the language fits the relationship |
| Explaining why a client may react in a certain way | Making regulated or consequential recommendations |
| Rewriting a draft for a specific context | Editing and sending the final message |
We deliberately did not position the model as an autonomous closer or a replacement for adviser judgment. The useful product was a supervised rehearsal and decision-support loop.
What changed after feedback
1. Start with the situation, not a generic chat
The first role-play assumed a client was already ready to buy. An adviser said that was unlike the conversations he actually faced. The next round used a more probing opening designed to discover what the client was looking for.
2. Explain the learning objective
Users could interpret an exaggerated bot response as bad simulation. The product needed to explain that the exercise was surfacing how a communication style might think underneath, even when a real person would not state it so directly.
3. Connect insight to an explicit next action
The UAT backlog prioritised clearer sequencing between analysis, recommended action, role-play, and message generation. It also captured practical changes: clearer client-specific labels, shorter responses, better defaults, search, progress feedback, and less ambiguous onboarding copy.
4. Design for the actual operating environment
Privacy questions, mobile usage, internal CRMs, spreadsheets, and WhatsApp were not edge cases. They were part of the system boundary. Any production version had to reduce copying, preserve context, and keep a human approval point before client communication.
The second-round signal
In a follow-up role-play session, the same adviser initially resisted the exercise because the responses felt artificial. As the conversation developed, he recognised patterns from an existing client and found the training value clearer.
That did not prove product-market fit or a company-wide deployment. It did validate a more precise product thesis: role-play was most useful when it was grounded in a recognisable situation, framed as deliberate practice, and aimed at users who had not yet built intuition from hundreds of real conversations.
Outcome
The defensible outcome was a better-scoped product and a concrete iteration backlog, not a manufactured revenue claim.
- We moved from “AI personality insight for every adviser” toward a clearer coaching workflow for newer advisers and team leads.
- We identified the handoff from insight → recommended action → practice → editable message as the core product sequence.
- We separated model capability from workflow adoption: better output alone could not fix unclear onboarding, missing context, or too many steps.
- We retested the hardest part of the experience with the same user and observed the conditions under which resistance turned into perceived value.