14 papers
On the use of foundation models in cognitive science
Raj Sanjay Shah, Alex Warstadt, Michael Frank +1
A host of recent studies have evaluated the cognitive and developmental alignment of Foundation Models (FMs). These investigations include evaluations of their correspondence to ad…
Human-Like Anaphor Resolution in Large Language Models
Keane Zhang, Varshini Chinta, Raj Sanjay Shah +1
Anaphors are expressions that refer to other expressions, called antecedents. The process of connecting the two is called resolution. Cognitive science has identified multiple fact…
Responsible Evaluation of AI for Mental Health
Hiba Arnaout, Anmol Goel, H. Andrew Schwartz +13
Although artificial intelligence (AI) shows growing promise for mental health care, current approaches to evaluating AI tools in this domain remain fragmented and poorly aligned wi…
When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations
Harsh Nishant Lalai, Raj Sanjay Shah, Hanspeter Pfister +2
Visualizations help communicate data insights, but deceptive data representations can distort their interpretation and propagate misinformation. While recent Vision Language Models…
The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning
Raj Sanjay Shah, Jing Huang, Keerthiram Murugesan +2
Unlearning in Large Language Models (LLMs) aims to enhance safety, mitigate biases, and comply with legal mandates, such as the right to be forgotten. However, existing unlearning…
Can LLM-Simulated Practice and Feedback Upskill Human Counselors? A Randomized Study with 90+ Novice Counselors
Ryan Louie, Raj Sanjay Shah, Ifdita Hasan Orney +3
The growing demand for accessible mental health support requires training more counselors, yet existing approaches remain resource-intensive and difficult to scale. LLMs can realis…