Microsoft · 2024–25
The question
Microsoft had conducted several studies about Copilot and AI use across Outlook, but the findings had not been synthesized into a shared product perspective. I reviewed a year of internal research, public conversations, and academic literature to develop a white paper on user trust and product priorities.
My role
UX Researchercontract, via Kadence
Timeline
Dec 2024 – Feb 2025paper delivered Feb 2025
Inputs
A year of internal studiesplus public social listening and academic literature
Deliverable
A point-of-view white paperwith design guidelines and a feature checklist
Why it mattered
Copilot was expanding across Microsoft 365, while research remained organized around individual features. Teams had separately studied drafting, inbox prioritization, scheduling, summaries, and automation. Across these studies, similar questions emerged about trust, user control, proactive assistance, and automation. I synthesized the findings into a shared point of view on user needs and product priorities for Copilot in Outlook.
What was needed
How should AI enter the email ecosystem without breaking trust?
A point of view on product priorities, the information and controls users need, and the requirements for more autonomous assistance.
Method
The synthesis at a glance
Inputs
Primary
12+ internal studies
drafting, prioritization, time-away, concept focus groups, workflow, out-of-office
Validation
Public social listening
used to assess themes identified in internal research
Grounding
Prior literature & internal frameworks
Analysis
Thematic synthesis and framework development
Outputs
A trust framework
transparency, control, learnability, staged autonomy
A pre-ship checklist
seven checklist categories a team runs before anything ships
Per-feature adoption guidance
criteria for assigning capabilities to each stage of autonomy
I extracted findings, open questions, participant concerns, and recommendations from each report into a shared repository. Each note retained its original source.
For each finding, I reviewed the original study, participant sample, research questions, and strength of support. Findings from small samples or exploratory studies were retained with those limitations documented.
I compared findings across studies with different research objectives. Research on inbox prioritization, drafting, out-of-office catch-up, project workflows, and automation identified recurring needs related to workload, transparency, correction, and control.
I compared themes from the internal research with public conversations about AI assistants and Microsoft 365 Copilot. Public posts were used as a secondary check on the synthesis and were not treated as participant research.
X · Microsoft 365
The Teams call summary is a genuine win. This is the kind of thing people actually asked for.
ZDNet
To a lot of users, the 365 Copilot launch read like a mess.
Reddit · r/sysadmin
Half the threads were people asking how to switch it off, not how to use it.
Threads · LinkedIn
Forced opt-in was the fastest way to make people resent a feature that might have helped them.
Paraphrased public posts reviewed during the synthesis.
Synthesis
Across the 12 studies, participants described related needs for contextual assistance, support with routine work, and visibility into AI decisions. I organized these findings around a chief-of-staff model, which gave teams a shared way to evaluate capabilities across Outlook.
How it was being built
Each one shipped and measured on its own, by the team that owned it.
What the paper argued for
One assistant that behaves like a chief of staff, working in your flow and on your behalf.
The research also showed a need for continuity across Mail, Calendar, Chat, and Docs. Separate memory in each product would require users to repeatedly provide the same context and preferences. The paper therefore recommended a consistent approach to context across Outlook experiences.
On-demand
Proactive
Automatic
01
The user asks; the assistant answers; the user reviews the output before using it.
Examples included summarizing a thread, drafting a reply, and finding an email. Across the studies, participants were most comfortable with this level of assistance.
02
The assistant surfaces summaries, suggestions, and priorities without being asked, but every action still belongs to the user.
At this stage, the assistant must decide what is worth surfacing and explain why. Irrelevant or unexplained suggestions could be perceived as noise.
03
The assistant acts on its own, sending, archiving, rescheduling, within boundaries the user has set.
The paper recommended moving a task to this stage only after users had experience with its on-demand and proactive versions.
The paper proposed introducing assistance in stages so users could understand and evaluate a capability before it became more autonomous.
The framework
The vision it all sits under
Copilot as a chief of staff
Show what Copilot looked at, why it made a recommendation, and how the user can correct or limit it in the moment.
Improve from user corrections and carry that learning across sessions, so Copilot does not feel like it resets every time it is opened.
Get the basics right first: names, dates, people, deadlines, and context. A confident wrong answer is worse than an incomplete one.
Staged autonomy
Move from on-demand to proactive to automatic assistance one task at a time, based on how users evaluate each workflow.
Across the studies, problems with trust were often associated with limited transparency, control, or learnability.
"What did it look at, and why did it suggest that?"
Users were more willing to evaluate Copilot's output when they could see the sources and reasoning behind it. Without that visibility, even useful suggestions could feel arbitrary.
"Can I correct it, limit it, or undo it here?"
Control had to exist at the moment of use. A setting buried elsewhere did not help when Copilot made a questionable judgment in the inbox.
"Does it get better after I correct it?"
Users did not expect Copilot to be perfect. They expected it to learn. Repeated mistakes were especially damaging because they made correction feel pointless.
Concept example
The example shows how explanations can help users evaluate AI-generated priorities.
Why: from your direct manager, contains a deadline today, and you replied to this thread twice this week.
Why: bulk promotional sender. You've never opened the previous 14 emails from this address.
Why: direct question addressed to you, from a close collaborator, on a document you edited yesterday.
Expected user response
"It marked something high priority. I have no idea why, so I'll check everything myself anyway."
Design recommendation: explain why the system assigned each priority.
The deliverable
The paper documented the research sources, synthesis, framework, design guidelines, and feature checklist for product teams.
The trust & safety checklist
The white paper included a pre-ship checklist for Copilot feature teams. Before making a capability more proactive or automatic, teams could assess whether users understood it, could correct or limit it, and could recover from mistakes.
The framework assumes a good-faith user and an honest environment. It does not cover adversarial use: an assistant that sends and replies on someone's behalf is also a new surface for impersonation and phishing, and a system that learns from corrections can be deliberately mistrained. Those questions were outside this paper's scope.
Impact
The paper was shared with relevant product teams, and the checklist was adopted as a review template for Copilot features.