Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

Abstract

Purpose: Manual feedback from senior surgeons observing less experienced trainees is a laborious task that is very expensive, time-consuming and prone to subjectivity. With the number of surgical procedures increasing annually, there is an unprecedented need to provide an accurate, objective and automatic evaluation of trainees' surgical skills in order to improve surgical practice. Methods: In this paper, we designed a convolutional neural network (CNN) to classify surgical skills by extracting latent patterns in the trainees' motions performed during robotic surgery. The method is validated on the JIGSAWS dataset for two surgical skills evaluation tasks: classification and regression. Results: Our results show that deep neural networks constitute robust machine learning models that are able to reach new competitive state-of-the-art performance on the JIGSAWS dataset. While we leveraged from CNNs' efficiency, we were able to minimize its black-box effect using the class activation map technique. Conclusions: This characteristic allowed our method to automatically pinpoint which parts of the surgery influenced the skill evaluation the most, thus allowing us to explain a surgical skill classification and provide surgeons with a novel personalized feedback technique. We believe this type of interpretable machine learning model could integrate within "Operation Room 2.0" and support novice surgeons in improving their skills to eventually become experts.

arXiv commentAccepted at IJCARS Special Issue for MICCAI 2018

References in corpus (4)

• Surgical Data Science: Enabling Next-Generation Surgery
• Transfer learning for time series classification
• Deep Learning with Convolutional Neural Network for Objective Skill Evaluation in Robot-assisted Surgery
• Evaluating surgical skills from kinematic data using convolutional neural networks

Cited by in corpus (7)

• A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directions
• Analysis of Executional and Procedural Errors in Dry-lab Robotic Surgery Experiments
• Robust Explainer Recommendation for Time Series Classification
• Think Step by Step: Chain-of-Gesture Prompting for Error Detection in Robotic Surgical Videos
• Machine Learning-Based Automated Assessment of Intracorporeal Suturing in Laparoscopic Fundoplication
• Understanding whole-body inter-personal dynamics between two players using neural Granger causality as the explainable AI (XAI)
• ReCAP: Recursive Cross Attention Network for Pseudo-Label Generation in Robotic Surgical Skill Assessment