Evaluating surgical skills from kinematic data using convolutional neural networks

Abstract

The need for automatic surgical skills assessment is increasing, especially because manual feedback from senior surgeons observing junior surgeons is prone to subjectivity and time consuming. Thus, automating surgical skills evaluation is a very important step towards improving surgical practice. In this paper, we designed a Convolutional Neural Network (CNN) to evaluate surgeon skills by extracting patterns in the surgeon motions performed in robotic surgery. The proposed method is validated on the JIGSAWS dataset and achieved very competitive results with 100% accuracy on the suturing and needle passing tasks. While we leveraged from the CNNs efficiency, we also managed to mitigate its black-box effect using class activation map. This feature allows our method to automatically highlight which parts of the surgical task influenced the skill prediction and can be used to explain the classification and to provide personalized feedback to the trainee.

arXiv commentAccepted at MICCAI 2018

References in corpus (2)

• Adam: A Method for Stochastic Optimization
• Surgical Data Science: Enabling Next-Generation Surgery

Cited by in corpus (17)

• Deep learning for time series classification: a review
• Video-based surgical skill assessment using 3D convolutional neural networks
• Adversarial Attacks on Deep Neural Networks for Time Series Classification
• Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks
• Deep Neural Network Ensembles for Time Series Classification
• Gait recognition via deep learning of the center-of-pressure trajectory
• dCAM: Dimension-wise Class Activation Map for Explaining Multivariate Data Series Classification
• Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field
• Deep learning for time series classification
• Live image-based neurosurgical guidance and roadmap generation using unsupervised embedding
• Deep Neural Networks for the Assessment of Surgical Skills: A Systematic Review
• Automatic alignment of surgical videos using kinematic data
• What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment
• Condition Assessment of Stay Cables through Enhanced Time Series Classification Using a Deep Learning Approach
• Explaining Regression Based Neural Network Model
• Recurrent and Spiking Modeling of Sparse Surgical Kinematics
• Towards Unified Surgical Skill Assessment