01. Project Overview
MyFLS (Fundamentals of Laparoscopic Surgery)
Laparoscopic surgery is a procedure performed in the tummy area using small incisions with the aid of a camera. Before residents perform surgery on patients, they need to pass a test called the Fundamentals of Laparoscopic Surgery (FLS). This project aims to improve the training experience of the FLS test for surgical residents.
02. Define the Problem
What are the challenges for surgical residents to practice FLS?
When surgical residents practice the FLS test, they don’t know where they did wrong. Typically after the initial walkthrough, no one is there to look over their shoulder to correct the way they move the tool. They repeat mistakes for months without timely feedback.
How might we improve the FLS training experience for surgical residents?
03. Solution
MyFLS
MyFLS, an integrated software system that provides real-time and post-performance feedback on surgical residents' FLS skill.
Feature 01
Real-time AI Feedback
The system can detect incorrect move and provide real-time AI feedback and show the correct move for residents to follow on the screen.
Feature 02
Post-performance Statistics
The tool provides statistics and summary overtime. Trainees can track their progress overtime.
Feature 03
Recording and Timer
Trainees can record their practice session for them to identify mistakes. An optional timer allows user to time themselves.
Feature 04
Learning Center
Trainers can upload learning materials to the platform where trainees can watch unlimited times.
Location:
University of Michigan Clinical Simulation Center
Observation date & time:
Mon Sep 19, 2022, 4pm - 5:30pm
04. Field Observation
What does the practice environment look like?
Equipment room is open and well-lit. The FLS training station is placed against the wall, consisting of a computer monitor, a white trainer box in the middle that contains all of the materials needed to perform the task and a camera inside the white box. The monitor plays what the camera captures in real time. No feedback system showing if trainees' movement is right or wrong.
Findings:
05. Literature Review
What are some existing products to address the "lack of feedback" in surgical training?
Findings:
AI evaluation tool sounds like a promising solution to addressing the unmet needs for feedback on performance. But can we make the feedback more real-time instead of just post-performance feedback?
06. Competitive Analysis
What are some solutions provided by other training centers to address this "lack of feedback" challenge?
In addition to different tools, we conducted a competitive analysis with 5 other medical schools and training centers and found that:
Findings
In previous literature review, we found VR simulators can create a false sense of game, but additional FLS curriculum as adopted by other training centers might sound like a promising solution for residents to get feedback.
07. Ideation
What are some possible solutions to address the user needs?
Based on all of the research so far, we excluded the solution of VR simulator (feels like a game) and identified the following three options: sensory gloves that provide real-time haptic feedback to hand movement, AI software system that provides real-time feedback, and additional FLS curriculum for trainer and trainees to meet and provide feedback. We used QOC - question, options and criteria diagram to rank these choices in the following:
AI software system ranked top as it can address the unmet needs for immediate feedback, shortage of trainers and yet be cost-effective.
Result:
08. User interviews
Dive deeper into user pain points.
We conducted 4 user interviews with surgical residents and 1 expert interview with a former FLS trainer and now Director of Learning Sciences at the Michigan Medicine Clinical Simulation Center. The followings are some key painpoints.
09. Low-fi prototype
Time to sketch ideas!
Based on user interviews, we sketched out some ideas like "AI feedback", "recording performance" to address user pain points.
10. Hi-fi Prototype
Okay, let's get real!
After some mini-usability testing sessions, we made iterations based on the feedback we received. The following are some of the key screens of hi-fidelity prototypes we developed.
Key Screens 01: Onboarding
During the onboarding, user can select their role as trainer or trainee, then they will be given a short intro video on what is "AI feedback", offered options whether to turn it on etc.
Key Screens 04: Progress Dashboard
Key Screens 05: Additional Human Feedback
The system provides user's performance statistics overtime. Trainees can track their progress on all tasks or on a specific task over time.
When trainers log into their system to review their trainee's performance, they can add time stamps to provide additional feedback to complement the AI feedback for their trainee.
Key Screens 02: AI Feedback
The AI feedback feature is turned on by default. When the system detects user's incorrect grasp movement, it will give real-time feedback and instruction for user to follow and correct their movement.
Key Screens 03: Recording and Sending for Review
Users can record their practice session. After the session is completed, they can send the recording to their attending/ trainer for more feedback. The recording is saved to the cloud and users can watch and review it at anytime they want.
11. Usability Testing
What do we hear from users?
Results
● Tested our product with 10 users (lay people with no FLS background).
● Observed the user while performing the task
● Initial test results, 78% of users were able to perform tasks as required
● Received suggestion on improving UI
● Updated the changes in the product for efficient usability.
Summary
12. Reflection
What can be done differently and what are some next steps?
• If given more time, ideally user testing needs to be done with real residents and trainers
• Invite experts to provide meaningful feedback on “AI feedback” portion
• Test out the design of AI feedback with real-world expert
• Keep updated on literature and latest progress of technology
13. Reference
Reviewed Research
1. Peters, J. H., Fried, G. M.,Swanstrom, L. L., Soper, N. J., Sillin, L. F., & Hoffman, K. (2004).Development and validation of a comprehensive program of education andassessment of the basic fundamentals of laparoscopic surgery. SurgeryJournal. 135(1), 21-27 (2004). https://doi.org/10.1016/S0039-6060(03)00156-92.
2. Sbernini, L., Quitadamo, F., Riillo, N., Lorenzo, A., Gaspari, L., & Saggio, G. (2018) Sensory-Glove-BasedOpen Surgery Skill Evaluation. IEEE Transactions on Human-MachineSystems. 48(2), 213-218. http://doi.org/10.1109/THMS.2017.27766033.
3. Coxworth, Ben. (2022). Smart surgicalgloves track medical students' hand movements. New Atlas.https://newatlas.com/wearables/smart-surgical-gloves-medical-students-hand-movements/4.Islam,
4. G., Kahol, K., Li, B., Smith, M., & Patel, V. L. (2016). Affordable,web-based surgical skill training and evaluation tool. Journal of biomedicalinformatics, 59, 102–114. https://doi.org/10.1016/j.jbi.2015.11.0025.
5. Blumstein, G. (2019). Research: Howvirtual reality can help train surgeons. Harvard Business Review.hbr.org/2019/10/research-how-virtual-reality-can-help-train-surgeons
6. Oropesa, I., Sánchez-González, P., Lamata, P., Chmarra, M. K., Pagador, J. B., Sánchez-Margallo, J. A.,Sánchez-Margallo, F. M., & Gómez, E. J. (2011). Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery. The Journal of surgical research, 171(1), e81–e95. https://doi.org/10.1016/j.jss.2011.06.034
7. Edelman, D.A., Mattos, M.A., & Bouwman, D.L. (2012). Value offundamentals of laparoscopic surgery training in a fourth-year medical schooladvanced surgical skills elective. Journal of Surgical Research, 177(2), 207-210. https://doi.org/10.1016/j.jss.2012.05.015