124 citations · 345 across the 33 of their papers we have counts for
20 papers · 1 filter
Towards AI-Assisted Clinical Trial Matching: Practical Considerations, Multicenter Evaluation, and Real-World Deployment
Yin Fang, Qiao Jin, Shubo Tian +24
Clinical trials are essential for advancing cancer care and drug development, but many fail because of insufficient patient enrollment. While there is growing interest in using AI…
What Do Biomedical NER and Entity Linking Benchmarks Measure? A Corpus-Centric Diagnostic Framework
Robert Leaman, Rezarta Islamaj, Zhiyong Lu
Biomedical named entity recognition (NER) and entity linking (EL) strongly depend on annotated corpora, but the utility of these resources for benchmarking is often assumed rather…
MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answering
Rezarta Islamaj, Robert Leaman, Joey Chan +13
Evaluating large language models (LLMs) in the biomedical domain requires benchmarks that can distinguish reasoning from pattern matching and remain discriminative as model capabil…
Overview of the MedHopQA track at BioCreative IX: track description, participation and evaluation of systems for multi-hop medical question answering
Rezarta Islamaj, Joey Chan, Robert Leaman +13
Multi-hop question answering (QA) remains a significant challenge in the biomedical domain, requiring systems to integrate information across multiple sources to answer complex que…
MedCite: Can Language Models Generate Verifiable Text for Medicine?
Xiao Wang, Mengjue Tan, Qiao Jin +5
Existing LLM-based medical question-answering systems lack citation generation and evaluation capabilities, raising concerns about their adoption in practice. In this work, we intr…
Cell-o1: Training LLMs to Solve Single-Cell Reasoning Puzzles with Reinforcement Learning
Yin Fang, Qiao Jin, Guangzhi Xiong +6
Cell type annotation is a key task in analyzing the heterogeneity of single-cell RNA sequencing data. Although recent foundation models automate this process, they typically annota…
Benchmarking Retrieval-Augmented Generation for Chemistry
Xianrui Zhong, Bowen Jin, Siru Ouyang +5
Retrieval-augmented generation (RAG) has emerged as a powerful framework for enhancing large language models (LLMs) with external knowledge, particularly in scientific domains that…
Supervising the search process produces reliable and generalizable information-seeking agents
Guangzhi Xiong, Qiao Jin, Xiao Wang +9
Large language models (LLMs) are transforming web search by shifting from document ranking to synthesizing answers, and are increasingly deployed as autonomous agentic search syste…
Ensuring Safety and Trust: Analyzing the Risks of Large Language Models in Medicine
Yifan Yang, Qiao Jin, Robert Leaman +15
The remarkable capabilities of Large Language Models (LLMs) make them increasingly compelling for adoption in real-world healthcare applications. However, the risks associated with…
Evaluating the Impact of Lab Test Results on Large Language Models Generated Differential Diagnoses from Clinical Case Vignettes
Balu Bhasuran, Qiao Jin, Yuzhang Xie +6
Differential diagnosis is crucial for medicine as it helps healthcare providers systematically distinguish between conditions that share similar symptoms. This study assesses the i…
Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions
Guangzhi Xiong, Qiao Jin, Xiao Wang +3
The emergent abilities of large language models (LLMs) have demonstrated great potential in solving medical questions. They can possess considerable medical knowledge, but may stil…
Closing the gap between open-source and commercial large language models for medical evidence summarization
Gongbo Zhang, Qiao Jin, Yiliang Zhou +11
Large language models (LLMs) hold great promise in summarizing medical evidence. Most recent studies focus on the application of proprietary LLMs. Using proprietary LLMs introduces…
MedCalc-Bench: Evaluating Large Language Models for Medical Calculations
Nikhil Khandekar, Qiao Jin, Guangzhi Xiong +14
As opposed to evaluating computation and logic-based reasoning, current benchmarks for evaluating large language models (LLMs) in medicine are primarily focused on question-answeri…
Accelerating Clinical Evidence Synthesis with Large Language Models
Zifeng Wang, Lang Cao, Benjamin Danek +3
Synthesizing clinical evidence largely relies on systematic reviews of clinical trials and retrospective analyses from medical literature. However, the rapid expansion of publicati…
Benchmarking Retrieval-Augmented Generation for Medicine
Guangzhi Xiong, Qiao Jin, Zhiyong Lu +1
While large language models (LLMs) have achieved state-of-the-art performance on a wide range of medical question answering (QA) tasks, they still face challenges with hallucinatio…
AgentMD: Empowering Language Agents for Risk Prediction with Large-Scale Clinical Tool Learning
Qiao Jin, Zhizheng Wang, Yifan Yang +8
Clinical calculators play a vital role in healthcare by offering accurate evidence-based predictions for various purposes such as prognosis. Nevertheless, their widespread utilizat…
Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports
Qingqing Zhu, Xiuying Chen, Qiao Jin +6
In radiology, Artificial Intelligence (AI) has significantly advanced report generation, but automatic evaluation of these AI-produced reports remains challenging. Current metrics,…
Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation
Yifan Yang, Xiaoyu Liu, Qiao Jin +2
Large language models like GPT-3.5-turbo and GPT-4 hold promise for healthcare professionals, but they may inadvertently inherit biases during their training, potentially affecting…
Quality of Answers of Generative Large Language Models vs Peer Patients for Interpreting Lab Test Results for Lay Patients: Evaluation Study
Zhe He, Balu Bhasuran, Qiao Jin +6
Lab results are often confusing and hard to understand. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to get their questions answered. We…
PubTator 3.0: an AI-powered Literature Resource for Unlocking Biomedical Knowledge
Chih-Hsuan Wei, Alexis Allot, Po-Ting Lai +7
PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for…