OperA: Attention-Regularized Transformers for Surgical Phase Recognition
arXiv:2103.03873 · doi:10.1007/978-3-030-87202-1_58
Abstract
In this paper we introduce OperA, a transformer-based model that accurately predicts surgical phases from long video sequences. A novel attention regularization loss encourages the model to focus on high-quality frames during training. Moreover, the attention weights are utilized to identify characteristic high attention frames for each surgical phase, which could further be used for surgery summarization. OperA is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos, outperforming various state-of-the-art temporal refinement approaches.
arXiv comment10 pages, 3 figures
References in corpus (7)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Language Models are Few-Shot Learners
- Zero-Shot Text-to-Image Generation
- TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks
- Video-based surgical skill assessment using 3D convolutional neural networks
- Less is More: Surgical Phase Recognition with Less Annotations through Self-Supervised Pre-training of CNN-LSTM Networks
- Surgical Data Science: A Consensus Perspective