File 02
2024
Designing trust into an AI assistant for clinicians
Sector
Stealth health-AI lab
Scale
Pilot in 3 hospital networks
Role
Staff Product Designer
Team
1 PM · 8 Eng · 2 ML · 1 Clinician
Duration
10 months
Client
30-second read
For hiring teams
Made every AI-drafted sentence traceable to its source in one tap.
Replaced confidence percentages with plain-language certainty, which clinicians trusted more.
Time to finish a note down roughly 40% in the pilot.
All interface visuals are abstracted. The pilot protocol and interface specs are not shareable.
Fig. 1 — Abstracted interface
Context
Clinicians were open to AI drafts but would not sign anything they couldn’t verify. Early prototypes with “92% confidence” labels made trust worse, not better.
Constraints
—
Patient data could never leave the hospital network.
—
Every claim needed an audit trail.
—
Clinicians had about 90 seconds between patients.
Decisions
Decision
Show sources inline or on demand?
A sources panel · Footnotes at the end
Rejected
Inline markers, source opens on tap
Chosen
Verification had to cost less than retyping the note.
Decision
Numbers or words for confidence?
Percentages · Colour-coded scores
Rejected
Three plain-language levels
Chosen
In testing, percentages read as false precision. Words read as judgement.
Decision
Can the AI send anything?
Auto-send low-risk notes
Rejected
Draft only, the clinician always signs
Chosen
It kept liability with the human and made adoption politically possible.
Outcome
~40%
Faster note completion in the pilot
3 / 3
Hospital networks continued after the trial
0
Notes sent without a clinician’s signature
Reflection
We tested with enthusiastic early adopters for too long. The skeptics gave us the design.
Full file
File 02 — Locked
Clinician interview synthesis
Confidence-language study
Audit trail specification