Interpretable ICD Code Embeddings with Self- and Mutual-Attention Mechanisms

Abstract

We propose a novel and interpretable embedding method to represent the international statistical classification codes of diseases and related health problems (i.e., ICD codes). This method considers a self-attention mechanism within the disease domain and a mutual-attention mechanism jointly between diseases and procedures. This framework captures the clinical relationships between the disease codes and procedures associated with hospital admissions, and it predicts procedures according to diagnosed diseases. A self-attention network is learned to fuse the embeddings of the diseases for each admission. The similarities between the fused disease embedding and the procedure embeddings indicate which procedure should potentially be recommended. Additionally, when learning the embeddings of the ICD codes, the optimal transport between the diseases and the procedures within each admission is calculated as a regularizer of the embeddings. The optimal transport provides a mutual-attention map between diseases and the procedures, which suppresses the ambiguity within their clinical relationships. The proposed method achieves clinically-interpretable embeddings of ICD codes, and outperforms state-of-the-art embedding methods in procedure recommendation.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• A Structured Self-attentive Sentence Embedding
• Attention-based Deep Multiple Instance Learning
• Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
• An Empirical Evaluation of Deep Learning for ICD-9 Code Assignment using MIMIC-III Clinical Notes
• Gromov-Wasserstein Learning for Graph Matching and Node Embedding
• Self-Attentive Sequential Recommendation
• Multi-Label Classification of Patient Notes a Case Study on ICD Code Assignment
• Distilled Wasserstein Learning for Word Embedding and Topic Modeling
• Learning Wasserstein Embeddings
• Embedding Complexity In the Data Representation Instead of In the Model: A Case Study Using Heterogeneous Medical Data