1.7k citations · 1.7k across the 28 of their papers we have counts for
31 papers
Clinician use of language models diverges from how the models are evaluated
Krithik Vishwanath, Haitong Lin, Anton Alyakin +19
Large language model (LLM) assistants are being deployed to clinicians across health systems, and judgments about their readiness rest largely on benchmark scores, most of them der…
Large language models are vulnerable to incidental information in clinical documentation and reasoning
Krithik Vishwanath, Brandon Ye, Anton Alyakin +4
Large language models (LLMs) are increasingly relied upon to support ambient documentation and clinical reasoning. Here we examine the impact of a failure mode shared between these…
Reason in Style: Discovering and Controlling Style in Language Models
Ioana Marinescu, Eric Karl Oermann, Kyunghyun Cho
Language models learn content and style jointly, making stylistic variation in their outputs difficult to identify and control. We study whether recurring styles in model responses…
Multi-Task Bayesian In-Context Learning
Qingyang Zhu, Eric Karl Oermann, Kyunghyun Cho
Bayesian predictive inference provides a principled framework for uncertainty quantification, data efficiency, and robust generalization. However, exact inference is often intracta…
Paradox of De-identification: A Critique of HIPAA Safe Harbour in the Age of LLMs
Lavender Y. Jiang, Xujin Chris Liu, Kyunghyun Cho +1
Privacy is a human right that sustains patient-provider trust. Clinical notes capture a patient's private vulnerability and individuality, which are used for care coordination and…
Large Language Models Predict Functional Outcomes after Acute Ischemic Stroke
Anjali K. Kapoor, Anton Alyakin, Jin Vivian Lee +8
Accurate prediction of functional outcomes after acute ischemic stroke can inform clinical decision-making and resource allocation. Prior work on modified Rankin Scale (mRS) predic…
Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
Krithik Vishwanath, Mrigayu Ghosh, Anton Alyakin +3
Specialized clinical AI assistants are rapidly entering medical practice, often framed as safer or more reliable than general-purpose large language models (LLMs). Yet, unlike fron…
Generalist Foundation Models Are Not Clinical Enough for Hospital Operations
Lavender Y. Jiang, Angelica Chen, Xu Han +16
Hospitals and healthcare systems rely on operational decisions that determine patient flow, cost, and quality of care. Despite strong performance on medical knowledge and conversat…
On the Relationship Between the Choice of Representation and In-Context Learning
Ioana Marinescu, Kyunghyun Cho, Eric Karl Oermann
In-context learning (ICL) is the ability of a large language model (LLM) to learn a new task from a few demonstrations presented as part of the context. Past studies have attribute…
Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation
Radhika Dua, Young Joon, Kwon +9
Radiological imaging is central to diagnosis, treatment planning, and clinical decision-making. Vision-language foundation models have spurred interest in automated radiology repor…
Evaluating the performance and fragility of large language models on the self-assessment for neurological surgeons
Krithik Vishwanath, Anton Alyakin, Mrigayu Ghosh +5
The Congress of Neurological Surgeons Self-Assessment for Neurological Surgeons (CNS-SANS) questions are widely used by neurosurgical residents to prepare for written board examina…
Gateformer: Advancing Multivariate Time Series Forecasting through Temporal and Variate-Wise Attention with Gated Representations
Yu-Hsiang Lan, Eric K. Oermann
There has been a recent surge of interest in time series modeling using the Transformer architecture. However, forecasting multivariate time series with Transformer presents a uniq…
Medical large language models are easily distracted
Krithik Vishwanath, Anton Alyakin, Daniel Alexander Alber +3
Large language models (LLMs) have the potential to transform medicine, but real-world clinical scenarios contain extraneous information that can hinder performance. The rise of ass…
It is Too Many Options: Pitfalls of Multiple-Choice Questions in Generative AI and Medical Education
Shrutika Singh, Anton Alyakin, Daniel Alexander Alber +9
The performance of Large Language Models (LLMs) on multiple-choice question (MCQ) benchmarks is frequently cited as proof of their medical capabilities. We hypothesized that LLM pe…
BPQA Dataset: Evaluating How Well Language Models Leverage Blood Pressures to Answer Biomedical Questions
Chi Hang, Ruiqi Deng, Lavender Yao Jiang +4
Clinical measurements such as blood pressures and respiration rates are critical in diagnosing and monitoring patient outcomes. It is an important component of biomedical data, whi…
CNS-Obsidian: A Neurosurgical Vision-Language Model Built From Scientific Publications
Anton Alyakin, Jaden Stryker, Daniel Alexander Alber +29
General-purpose VLMs demonstrate impressive capabilities, but their opaque training on uncurated internet data poses critical limitations for high-stakes decision-making, such as i…
MedG-KRP: Medical Graph Knowledge Representation Probing
Gabriel R. Rosenbaum, Lavender Yao Jiang, Ivaxi Sheth +11
Large language models (LLMs) have recently emerged as powerful tools, finding many medical applications. LLMs' ability to coalesce vast amounts of information from many sources to…
MedMobile: A mobile-sized language model with clinical capabilities
Krithik Vishwanath, Jaden Stryker, Anton Alyakin +2
Language models (LMs) have demonstrated expert-level reasoning and recall abilities in medicine. However, computational costs and privacy concerns are mounting barriers to wide-sca…
HeadCT-ONE: Enabling Granular and Controllable Automated Evaluation of Head CT Radiology Report Generation
Julián N. Acosta, Xiaoman Zhang, Siddhant Dogra +5
We present Head CT Ontology Normalized Evaluation (HeadCT-ONE), a metric for evaluating head CT report generation through ontology-normalized entity and relation extraction. HeadCT…
Refining Packing and Shuffling Strategies for Enhanced Performance in Generative Language Models
Yanbing Chen, Ruilin Wang, Zihao Yang +2
Packing and shuffling tokens is a common practice in training auto-regressive language models (LMs) to prevent overfitting and improve efficiency. Typically documents are concatena…
Large-Scale Multi-omic Biosequence Transformers for Modeling Protein-Nucleic Acid Interactions
Sully F. Chen, Robert J. Steele, Glen M. Hocky +3
The transformer architecture has revolutionized bioinformatics and driven progress in the understanding and prediction of the properties of biomolecules. To date, most biosequence…
Generalization in Healthcare AI: Evaluation of a Clinical Large Language Model
Salman Rahman, Lavender Yao Jiang, Saadia Gabriel +3
Advances in large language models (LLMs) provide new opportunities in healthcare for improved patient care, clinical decision-making, and enhancement of physician and administrator…
Making the Most Out of the Limited Context Length: Predictive Power Varies with Clinical Note Type and Note Section
Hongyi Zheng, Yixin Zhu, Lavender Yao Jiang +2
Recent advances in large language models have led to renewed interest in natural language processing in healthcare using the free text of clinical notes. One distinguishing charact…
Language Model Classifier Aligns Better with Physician Word Sensitivity than XGBoost on Readmission Prediction
Grace Yang, Ming Cao, Lavender Y. Jiang +6
Traditional evaluation metrics for classification in natural language processing such as accuracy and area under the curve fail to differentiate between models with different predi…