1.7k citations · 1.7k across the 28 of their papers we have counts for
17 papers · 1 filter
Clinician use of language models diverges from how the models are evaluated
Krithik Vishwanath, Haitong Lin, Anton Alyakin +19
Large language model (LLM) assistants are being deployed to clinicians across health systems, and judgments about their readiness rest largely on benchmark scores, most of them der…
Large language models are vulnerable to incidental information in clinical documentation and reasoning
Krithik Vishwanath, Brandon Ye, Anton Alyakin +4
Large language models (LLMs) are increasingly relied upon to support ambient documentation and clinical reasoning. Here we examine the impact of a failure mode shared between these…
Reason in Style: Discovering and Controlling Style in Language Models
Ioana Marinescu, Eric Karl Oermann, Kyunghyun Cho
Language models learn content and style jointly, making stylistic variation in their outputs difficult to identify and control. We study whether recurring styles in model responses…
Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
Krithik Vishwanath, Mrigayu Ghosh, Anton Alyakin +3
Specialized clinical AI assistants are rapidly entering medical practice, often framed as safer or more reliable than general-purpose large language models (LLMs). Yet, unlike fron…
Generalist Foundation Models Are Not Clinical Enough for Hospital Operations
Lavender Y. Jiang, Angelica Chen, Xu Han +16
Hospitals and healthcare systems rely on operational decisions that determine patient flow, cost, and quality of care. Despite strong performance on medical knowledge and conversat…
On the Relationship Between the Choice of Representation and In-Context Learning
Ioana Marinescu, Kyunghyun Cho, Eric Karl Oermann
In-context learning (ICL) is the ability of a large language model (LLM) to learn a new task from a few demonstrations presented as part of the context. Past studies have attribute…
Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation
Radhika Dua, Young Joon, Kwon +9
Radiological imaging is central to diagnosis, treatment planning, and clinical decision-making. Vision-language foundation models have spurred interest in automated radiology repor…
Evaluating the performance and fragility of large language models on the self-assessment for neurological surgeons
Krithik Vishwanath, Anton Alyakin, Mrigayu Ghosh +5
The Congress of Neurological Surgeons Self-Assessment for Neurological Surgeons (CNS-SANS) questions are widely used by neurosurgical residents to prepare for written board examina…
Medical large language models are easily distracted
Krithik Vishwanath, Anton Alyakin, Daniel Alexander Alber +3
Large language models (LLMs) have the potential to transform medicine, but real-world clinical scenarios contain extraneous information that can hinder performance. The rise of ass…
It is Too Many Options: Pitfalls of Multiple-Choice Questions in Generative AI and Medical Education
Shrutika Singh, Anton Alyakin, Daniel Alexander Alber +9
The performance of Large Language Models (LLMs) on multiple-choice question (MCQ) benchmarks is frequently cited as proof of their medical capabilities. We hypothesized that LLM pe…
BPQA Dataset: Evaluating How Well Language Models Leverage Blood Pressures to Answer Biomedical Questions
Chi Hang, Ruiqi Deng, Lavender Yao Jiang +4
Clinical measurements such as blood pressures and respiration rates are critical in diagnosing and monitoring patient outcomes. It is an important component of biomedical data, whi…
MedMobile: A mobile-sized language model with clinical capabilities
Krithik Vishwanath, Jaden Stryker, Anton Alyakin +2
Language models (LMs) have demonstrated expert-level reasoning and recall abilities in medicine. However, computational costs and privacy concerns are mounting barriers to wide-sca…
Refining Packing and Shuffling Strategies for Enhanced Performance in Generative Language Models
Yanbing Chen, Ruilin Wang, Zihao Yang +2
Packing and shuffling tokens is a common practice in training auto-regressive language models (LMs) to prevent overfitting and improve efficiency. Typically documents are concatena…
Generalization in Healthcare AI: Evaluation of a Clinical Large Language Model
Salman Rahman, Lavender Yao Jiang, Saadia Gabriel +3
Advances in large language models (LLMs) provide new opportunities in healthcare for improved patient care, clinical decision-making, and enhancement of physician and administrator…
Making the Most Out of the Limited Context Length: Predictive Power Varies with Clinical Note Type and Note Section
Hongyi Zheng, Yixin Zhu, Lavender Yao Jiang +2
Recent advances in large language models have led to renewed interest in natural language processing in healthcare using the free text of clinical notes. One distinguishing charact…
Language Model Classifier Aligns Better with Physician Word Sensitivity than XGBoost on Readmission Prediction
Grace Yang, Ming Cao, Lavender Y. Jiang +6
Traditional evaluation metrics for classification in natural language processing such as accuracy and area under the curve fail to differentiate between models with different predi…
The Utility of General Domain Transfer Learning for Medical Language Tasks
Daniel Ranti, Katie Hanss, Shan Zhao +4
The purpose of this study is to analyze the efficacy of transfer learning techniques and transformer-based models as applied to medical natural language processing (NLP) tasks, spe…