4 papers
Evaluating Social Engineering Risks in AI-based Interaction using Biometrics and a Gaming Setup
Roberto Daza, Javier Irigoyen, Ivan Lopez +5
We introduce AIriskEval-gaming, an open platform and dataset to evaluate social engineering risks in LLM-mediated multimodal interaction through controlled games. It supports human…
"Are you an AI?" Analyzing Client Suspicion of AI Use in Crisis Counseling
Shreya Shah, Akshay Swaminathan, Meghana Simhadri +14
As artificial intelligence (AI) tools get increasingly deployed for mental healthcare, public trust in these systems remains uncertain. It is unclear how clients perceive AI involv…
FactEHR: A Dataset for Evaluating Factuality in Clinical Notes Using LLMs
Monica Munnangi, Akshay Swaminathan, Jason Alan Fries +8
Verifying and attributing factual claims is essential for the safe and effective use of large language models (LLMs) in healthcare. A core component of factuality evaluation is fac…
Distilling Large Language Models for Efficient Clinical Information Extraction
Karthik S. Vedula, Annika Gupta, Akshay Swaminathan +3
Large language models (LLMs) excel at clinical information extraction but their computational demands limit practical deployment. Knowledge distillation--the process of transferrin…