AI Uncertainty study

Project focused on improving AI-assisted auto-contouring of Organs at Risk (OAR) in radiation therapy.

Through user research, we explored clinicians’ trust in AI, how they prefer uncertainty to be communicated (numbers vs visuals, certainty vs uncertainty), and how much control they want over what information is shown.
                  

COMPANY Varian
(Siemens Healthineers)
www.varian.com

MY ROLE
Product Designer
Project Lead

 WHEN2024-25


A bit of context


In radiation therapy, automatic contouring of Organs at Risk (OAR) is essential for dose planning. Deep learning models used for OAR auto-contouring often encounteruncertainty, especially when handling medical images that differ from their training data (known as out-of-distribution (OOD) cases.)

Our current work aims to improve the OAR auto-contouring review workflow by detecting OOD cases and visualizing the AI model’s uncertainty and we asked users to help us create a better solution specifically when it comes to understand the level of uncertainty.

Research questions


  • What’s their general confidence / familiarity / trust with AI?
  • When it comes to the level of uncertainty: do they want to see numbers?
  • To which extent theywant to have control over what they can see?
  • Is it better to talk about uncertainty or certainty?





Approach 

Which steps did we take?



Let’s dig



STEP 1.
Stakeholders interviews and draft of the journey

To get a deeper understanding of the customer journey from an internal perspective, we conducted interviews with 10 key stakeholders across different teams—Marketing, Installation, Training & Education, and Customer Service.

Each stakeholder shared insights into their daily responsibilities, the challenges they face, and what aspects of their work run smoothly. They also contributed ideas and suggestions for improvement, giving us a well-rounded view of how things currently operate and where there’s room to enhance the experience.


STEP 2.
Survey based on Google HEART Framework

We surveyed 37 users (28 IDENTIFY customers, 9 competitor customers) across the US & EU using the Google HEART framework.

We focused on five key areas:
  • Happiness – Satisfaction, NPS, and usability.
  • Engagement – Usage patterns and full vs. partial adoption.
  • Adoption – Ease of clinical use, reliance on tattoos, and learnability.
  • Retention – Willingness to upgrade and use of advanced features.
  • Task Success – Ability to complete a full clinical workflow.

The survey provided clear insights into user needs, IDENTIFY’s impact, and areas for improvement.


STEP 3.
Interactive workshop: understanding the customer journey


To dive deeper into the user experience, we hosted an interactive workshop designed to map out the customer journey in a hands-on way. We brought together 30 users, including Radiotherapists and Medical Physicists, to gain insights from their real-world experiences.

Participants took part in three key exercises, each aimed at uncovering insights and opportunities for improvement:
  1. Mapping Roles – Identifying everyone involved at each stage of the journey to ensure a clear understanding of responsibilities and interactions.
  2. Experience Rating – Using a simple 1-5 scale with visual markers, participants rated the customer experience at different touchpoints, highlighting both strengths and areas that need attention.
  3. Improvement Discussion – A collaborative session where participants shared challenges, proposed solutions, and explored ideas to enhance the overall experience.

By engaging directly with those who interact with the system daily, we gained invaluable feedback that will help refine and optimize the customer journey for Radiotherapists, Medical Physicists, and beyond.
  




STEP 4.
Analyzing user feedback & key insights

The feedback collected was digitized and thoroughly analyzed, allowing us to identify key trends, user behaviors, and areas for improvement. By structuring the data, we gained clearer insights into how users interact with IDENTIFY, what works well, and where there’s room for enhancement. 



By organizing and presenting the insights in a clear and engaging way, we ensured that everyone could easily interpret the data, understand key takeaways, and use it to drive improvements.




STEP 5.
Presentation to stakeholders and next steps

We presented the customer journey to stakeholders, showcasing key insights and pain points. Following this, we worked closely with select teams to kickstart a structured approach to addressing user feedback.
(Currently on going.)