9 papers
Protecting Sensitive Data in Image Synthesis via PAC-Private Adaptation for Diffusion Models
Boming Miao, Tao Zhang, Netanel Raviv +3
Synthetic data are increasingly used as an alternative to sharing sensitive records. However, synthetic data generation does not guarantee privacy, as diffusion models trained or a…
RadOT-Eval: Auditable Structured-Evidence Transport for Radiology Report Evaluation
Weixin Liu, Juming Xiong, Yang Li +5
Automatic evaluation is critical for high-stakes text generation, where errors often involve omitted findings, hallucinated content, polarity reversals, location changes, uncertain…
MHGraphBench: Knowledge Graph-Grounded Benchmarking of Mental Health Knowledge in Large Language Models
Weixin Liu, Congning Ni, Shelagh A. Mulvaney +4
Large language models (LLMs) are increasingly used in the mental health domain, yet it remains unclear how well they capture related biomedical knowledge and how reliably they appl…
Blending Human and LLM Expertise to Detect Hallucinations and Omissions in Mental Health Chatbot Responses
Khizar Hussain, Bradley A. Malin, Zhijun Yin +2
As LLM-powered chatbots are increasingly deployed in mental health services, detecting hallucinations and omissions has become critical for user safety. However, state-of-the-art L…
Disentangling Prompt Element Level Risk Factors for Hallucinations and Omissions in Mental Health LLM Responses
Congning Ni, Sarvech Qadir, Bryan Steitz +14
Mental health concerns are often expressed outside clinical settings, including in high-distress help seeking, where safety-critical guidance may be needed. Consumer health informa…
Coverage-Controlled Preference Mining from Noisy Claim Verification for Evidence-Grounded Generation
Weixin Liu, Congning Ni, Qingyuan Song +4
Evidence-grounded generation produces summaries whose claims should be supported by supplied evidence, but claim-level verifiers provide noisy feedback and can reward models that s…
Judging with Confidence: Calibrating Autoraters to Preference Distributions
Zhuohang Li, Xiaowei Li, Chengyu Huang +11
The alignment of large language models (LLMs) with human values increasingly relies on using other LLMs as automated judges, or ``autoraters''. However, their reliability is limite…
SMOTE-DP: Improving Privacy-Utility Tradeoff with Synthetic Data
Yan Zhou, Bradley Malin, Murat Kantarcioglu
Privacy-preserving data publication, including synthetic data sharing, often experiences trade-offs between privacy and utility. Synthetic data is generally more effective than dat…
Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation
Zhuohang Li, Jiaxin Zhang, Chao Yan +4
Language models (LMs) are known to suffer from hallucinations and misinformation. Retrieval augmented generation (RAG) that retrieves verifiable information from an external knowle…