7 papers
The Mirage of LLM Guardrails: A Case Study in AI-Assisted Medical Note Manipulation
Davis Yadav, Amulya Yadav
The rapid deployment of large language models (LLMs) in healthcare settings makes the reliability of their built-in guardrails against malicious queries a question of urgent practi…
The Hardness of Achieving Impact in AI for Social Impact Research: A Ground-Level View of Challenges & Opportunities
Aditya Majumdar, Wenbo Zhang, Kashvi Prawal +1
AI for Social Impact (AI4SI) is an emergent field harnessing interdisciplinarities between the fields of artificial intelligence (AI), machine learning (ML), and the social science…
Designing User-Centric Metrics for Evaluation of Counterfactual Explanations
Firdaus Ahmed Choudhury, Ethan Leicht, Jude Ethan Bislig +2
Counterfactual Explanations (CFEs) have grown in popularity as a means of offering actionable guidance by identifying the minimum changes in feature values required to flip an ML m…
Evaluating Large Language Models on Rare Disease Diagnosis: A Case Study using House M.D
Arsh Gupta, Ajay Narayanan Sridhar, Bonam Mingole +1
Large language models (LLMs) have demonstrated capabilities across diverse domains, yet their performance on rare disease diagnosis from narrative medical cases remains underexplor…
CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback
Wenbo Zhang, Aditya Majumdar, Amulya Yadav
Large Language Models (LLMs) have demonstrated remarkable capabilities across various NLP tasks but struggle with code-mixed (or code-switched) language understanding. For example,…
Dr. GPT Will See You Now, but Should It? Exploring the Benefits and Harms of Large Language Models in Medical Diagnosis using Crowdsourced Clinical Cases
Bonam Mingole, Aditya Majumdar, Firdaus Ahmed Choudhury +3
The proliferation of Large Language Models (LLMs) in high-stakes applications such as medical (self-)diagnosis and preliminary triage raises significant ethical and practical conce…