3M X SCADpro Mixed Reality Tech Collaboration
Duration: 8 weeks
All Teammates: SCADpro team (14 people)
My Role: UX/UI Lead
Physician burnout is a highly ubiquitous problem in the healthcare space with over 50% of physicians in the United States reporting symptoms related to burnout. The key drivers of physician burnout spread wide and deep, ranging from administrative barriers and operational inefficiencies to information overload and data entry burden in physicians’ daily workflow. The current solutions are lacking in addressing this problem and offering solutions that enable physicians to spend less time with documentation and more quality time with patients.
This is project is protected by a non-disclosure agreement, hence everything revealed here is restricted to my role and work.
The Problem Statement / The Final Solution
The Hololens/MR Experience:
Solving for Physician Burnout
A need for a holistic and a human-centered design solution that would be focused towards making physicians work smarter.
How might we reduce physician burnout by integrating the use of AR/VR technology in their workflow as the technology develops over the next 5 years?
SOLUTION
Our prototype for the final solution consists of a collaborative effort between physicians and their team via a Mixed Reality HMD and a mobile device. In our prototype, we specifically used a HoloLens 2 and an iPad. The users can view 2D imaging slices of a 3D volume where they will be able to manipulate the slice layers on the axial, coronal, and sagittal planes. The user has the option to switch between viewing the 3D volume, 3D holographic heart, and an animated 3D digital twin. They can also toggle between different preset transfer settings as well as the standard color palettes for medical imaging. Different detail levels may also be selected.
When viewing the 3D volume, the user will be able to manipulate two different slice modes - box and slice cutters - allowing the physician to hone in on specific areas of the volume. Either in pre-op or operation work environments, the user can move in and out
of the volume via sliders to see each individual slice layer where they can then make a mark and leave an annotation for future reference. This mark and note will then be remapped from the 2D slice to the 3D volume. Along with the user in the HoloLens, the other team members may collaborate via an iPad to either watch or simultaneously interact with the updating interfaces.
CONCEPT DESIGN Hand Menu
The physician’s workload was divided into two areas: their basic, overarching needs and their specific needs for different medical environments. The global hand menu provides the necessary features for the physician’s daily work followed by environment-specific shortcuts as well. The ‘Schedules’, ‘Pagers’, and ‘Search’ features are specifically engineered to cater to the physician’s workflow, with additional customized buttons solely dedicated to spaces that match the physician’s movement across a medical facility.
CONCEPT DESIGN Medical Environments
Rounding
Pre-Op Meeting
OR
DESIGN GUIDELINES
DESIGN GUIDELINES Spatial Design Guide
Cognitive Aspects
The information hierarchy was structured based on these zones (Eye Focus Zone, Eye Comfort Zone, and the Neck Comfort Zone): the center contained the Work Zone around which important information was placed to support a physician’s tasks. CTA and other actions were placed in the Neck Comfort Zone.
Relaxed FOV of the Hololens
With respect to the ergonomics of the human eye, it was found that the human eye can comfortably look up and down in a range of up to 30 to 35 degrees horizontally and vertically. The outer region beyond the 35 degrees, was based on the movement of the human neck, which stretched up to 120 degrees.
Ergonomically, human heads tend to look down or face down about minus 10-15 degrees, while the eyes tend to look upward. Drawing from this, it was gathered that it would be most comfortable if the visual center (where work would be done) was placed about six degrees below the horizon.
DESIGN PROCESS
DESIGN PILLARS
Drawing from the research insights, three design pillars were built, to serve as guides for solution building; aspects, and traits that the solution would reflect.
EMPATHY
The first pillar focused on empathy. It was important to ensure that all the solutions were designed with empathy in mind, not just for physicians and healthcare providers, but also for the patients to ensure that they don’t feel overwhelmed with the vast amount of knowledge and medical jargon.
CONNECTION
The second pillar looked into connection. Solutions structured around this pillar would help healthcare staff within and across teams stay connected to help with better communication and organizational efficiency.
INTELLIGENCE
The third and last pillar ensured that the solution would be intelligent. This implied that each system offering would incorporate the relevant technology such as artificial intelligence, object recognition, and the internet of medical things (IoMT) to make the proposed concept a reality.
REQUIREMENTS
A product requirement analysis was conducted on the key offerings of the team’s proposed concept. The key features were plotted on a graph where the x-axis indicated the significance of the requirement while the y-axis marked the impact of the requirement. The design system’s product requirements were derived from the elements in the upper right quadrant.
Ability to display patient profiles
Ability to mark or draw on the 3D model
Ability to share the same virtual space
Allowing the physicians to share the same virtual information
Ability to convert voice input into documentation
USER JOURNEY MAPS
Rounding
The primary benefits that the solution provides would be to make use of spatial awareness to improve information access.
It would utilize AI-synthesised insights to reduce diagnostic errors and improve physicians’ efficiency as well as reduce cognitive load.
The solution would record key conversation snippets to facilitate simpler chart-taking post- rounding, enabling the doctor to build a rapport with patients by efficiently pulling and sharing the relevant information.
Spatially-prioritized information would ensure that information that is not looked at is still accessible to the physician if needed.
Pre-Op Meeting
The pre-op meeting environment would be focused on creating a cloud-based AI algorithm that provides physicians with a more accurate and effective search engine.
Misdiagnoses would be reduced by enabling more intuitive and accurate controls of scans in 3D spaces.
Efficiency would be improved by reducing
the need to get trained to use 2D-based scan viewing tools. The usability of six-degree-of- freedom scan manipulation would be enhanced, ensuring a higher quality system.Patient outcomes would improve by facilitating planning and notetaking, storing information in a cloud based storage, and providing physicians with an abundant medical academic database.
Operation Room
To facilitate a seamless medical experience in the Operation Room, a hands-free interaction would be employed to reduce the physician’s cognitive load, allowing them to focus on the patient.
A flexible visualization of element placement would provide customized preferences for each physician based on their necessities.
There would be an improved object detection to enhance the efficiency of operation preparation and minimize human errors.
Sharing the lead physician’s Point of View (PoV) would enhance streamlined communication and improve multidisciplinary collaboration.
TECH RESEARCH
Since the project brief focuses on leveraging AR/VR/MR technology, the team continued secondary research by looking into existing and emerging uses of different technologies and also considering pain points of these technologies in the healthcare field.
Through a thorough competitive analysis of existing technologies and products, potential considerations and opportunities were synthesized. Some concerns from this research include poor usability and redundancies of current products, difficulty with patient-physician communication and general limitations of mixed reality technologies. At the same time, opportunities are untapped, and there is scope for better system integration, and using mixed reality to visualize aggregated data better to improve patient-physician interactions.
The technological research findings are mapped out in a table that outlines current products, general workflow, existing features, pain points and areas of improvement for different technologies such as immersive technology, mobile applications, and Internet of Things.
SUPPORTING TECHNOLOGIES
Used to Implement these Features
Information Architecture
Global Navigations + Customized Button
The global hand menu provides necessary features for a physician’s daily work, and has shortcuts for each working environment. Schedules, pagers, and searches target the physicians' daily work, and customized buttons based on dedicated space provide shortcuts and automatically match the physicians’ needs in different working environments.
Information Architecture for Rounding
The structured and consistent patient profiles improve the communication among different health care teams while also bettering the efficiency
of patient rounding. Within the patient profiles, physicians can access patient information such as their diagnosis, current medications, charts and test results.
Information Architecture for Meeting
The customized button for the physician meeting environment provides a meeting mode for physicians. There are three main functions under the meeting mode: Sharing physician view virtually, turning 2D to 3D, and adding spatial annotations. This helps in facilitating smooth communication amongst physicians.
Information Architecture for OR
Instead of utilizing a fixed and set information architecture, voice recognition would provide enough flexibility to reduce physician workload.
StyleGuide
Slate Placement and Persistence
To account for cognitive load, the placement of information had to be intentional. Information that is uncalled for should not be shown unless it is a warning or priority notification. Slates should be placed next to the corresponding objects or around the patient, if possible.
The following types of slates were identified:
State Visualization
State visualization is necessary for cases that require object tracking.
Preview
To reiterate intentional placement of elements, allowing for a preview of the elements on patient cards with names and basic information, before the physician activates and expands on the card would help ease the cognitive load.
Informational
Heads-up displays are displayed as persistent elements to the physician’s head (head-locked) and will show succinct AI-synthesised information.
