Emergency Department Decision Support using Clinical Pseudo-notes

Abstract

In this work, we introduce the Multiple Embedding Model for EHR (MEME), an approach that serializes multimodal EHR tabular data into text using pseudo-notes, mimicking clinical text generation. This conversion not only preserves better representations of categorical data and learns contexts but also enables the effective employment of pretrained foundation models for rich feature representation. To address potential issues with context length, our framework encodes embeddings for each EHR modality separately. We demonstrate the effectiveness of MEME by applying it to several decision support tasks within the Emergency Department across multiple hospital systems. Our findings indicate that MEME outperforms traditional machine learning, EHR-specific foundation models, and general LLMs, highlighting its potential as a general and extendible EHR representation strategy.

References in corpus (17)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• On the Opportunities and Risks of Foundation Models
• A Survey of Large Language Models
• BERTopic: Neural topic modeling with a class-based TF-IDF procedure
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Towards Out-Of-Distribution Generalization: A Survey
• Learning from Few Examples: A Summary of Approaches to Few-Shot Learning
• Benchmarking emergency department triage prediction models with machine learning and large public electronic health records
• Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequences
• TabLLM: Few-shot Classification of Tabular Data with Large Language Models
• A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records
• LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks
• GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning
• ExBEHRT: Extended Transformer for Electronic Health Records to Predict Disease Subtypes & Progressions
• A scoping review of using Large Language Models (LLMs) to investigate Electronic Health Records (EHRs)
• A Data-Centric Approach To Generate Faithful and High Quality Patient Summaries with Large Language Models
• Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine