AR Surgery Guide App

This project involved developing an AR app named SKIA CMF(Cranio-Maxillofacial) for plastic surgeries, building upon the technology originally designed for breast surgery. The SKIA CMF app is an AR app that plastic surgeons use to perform facial surgery. The AR shows the location of the patient's blood vessels or lesions with a high degree of accuracy.

Background

As a lead product designer, my primary responsibility was to conduct in-depth user research to devise the strategic direction for this product. I then had to show a modern user interface and enhance its usability through usability testing, ensuring it meets the high standards required for use by surgeons.

Role

There was an existing AR surgical app for breast surgery, but it had an outdated UI that caused inconvenience for medical professionals using the app. To adapt this app for Cranio-Maxillo-Facial plastic surgery, I had to revise its UX/UI. As this app is used by medical professionals, it was crucial to enhance its usability, considering that its functionality can have a critical impact on patient safety. It was necessary to discover essential features of plastic surgery and incorporate them into an attractive UI.

App for breast surgery

Discover User Experience

1 Breast surgery app flow analysisTo design the CMF(Cranio-Maxillofacial) app, I first analyzed the existing breast surgery app. Organize and analyze the usage flow of the app.

2 Heuristic evaluationConducted heuristic evaluation using the breast app and found usability issues.

In-depth interviews were conducted with three medical staff who used the AR application under development for plastic surgery. I investigated the workflow of how to use the app in the surgical procedure and requested to perform the main tasks of the app. Through observation and interview, the needs and pain points of the medical staff using the app were discovered.

3 User research

I coded the findings of the research and analyzed them by drawing the affinity diagram. And determine the direction and required functions of the CMF(Cranio-Maxillofacial) plastic surgery AR app design.

Key Findings Through Discovery

1 Unclear AR menu icons

2 Unclear instructions and progress for using an AR function

3 Cumbersome mode-switching flow - Go back to the patient list screen to go to 3D mode or AR mode

4 A lesion control button and a lesion’s location aren’t intuitively matched - which lesion button to press to control it

5 Needed features: Screen capture, drawing function in AR mode, landmark feature in AR mode, adding CT information to the patient list, etc

Wireframe

Prototyping / Usability Testing

I made an interactive prototype and conducted usability testing of the designed UI with medical staff. And I was able to discover points that need to improve in the design.

Key Findings Through Usability Testing

1 The handle used to move the CT images should not obscure the surgical information.

2 It's hard to find the location of the landmark feature menu.

Impact

1 I won an award at the iF Design Awards 2023.

2 A paper on the design of this project, for which I am the lead author, was published in the Journal of the HCI Korea Society in 2022.

Takeaways

AR features aren't frequently used by most, especially medical professionals. This highlighted the importance of an intuitive UI and user-friendly UX for AR functionalities.

When designing a matching function between the 2D UI menu and objects within the 3D AR space, it can be challenging for users to discern which 2D UI button corresponds to an object in the 3D space. I learned that ensuring users easily recognize correlations is crucial.

While the medical field often prioritizes function over design, I've observed challenges with the app that have a poorly designed UI/UX. Conversely, although we designed a visually appealing CT image control handle, we had to replace it with a slider control due to concerns about obscuring medical information. Balancing function and design continues to be a significant consideration in medical applications.