46 citations · 86 across the 27 of their papers we have counts for
28 papers
Closing Ambient Clinical Documentation Gaps with Automated Provider Queries
Joseph Paul Cohen, Raj Shah, Han-Chin Shing +4
Provider queries are clarifying requests sent by clinical documentation specialists to physicians to close gaps in the clinical note and ensure accurate billing. Prior work automat…
On the use of foundation models in cognitive science
Raj Sanjay Shah, Alex Warstadt, Michael Frank +1
A host of recent studies have evaluated the cognitive and developmental alignment of Foundation Models (FMs). These investigations include evaluations of their correspondence to ad…
Human-Like Anaphor Resolution in Large Language Models
Keane Zhang, Varshini Chinta, Raj Sanjay Shah +1
Anaphors are expressions that refer to other expressions, called antecedents. The process of connecting the two is called resolution. Cognitive science has identified multiple fact…
When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations
Harsh Nishant Lalai, Raj Sanjay Shah, Hanspeter Pfister +2
Visualizations help communicate data insights, but deceptive data representations can distort their interpretation and propagate misinformation. While recent Vision Language Models…
The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning
Raj Sanjay Shah, Jing Huang, Keerthiram Murugesan +2
Unlearning in Large Language Models (LLMs) aims to enhance safety, mitigate biases, and comply with legal mandates, such as the right to be forgotten. However, existing unlearning…
BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop
Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul +7
The goal of the BabyLM is to stimulate new research connections between cognitive modeling and language model pretraining. We invite contributions in this vein to the BabyLM Worksh…
Responsible Evaluation of AI for Mental Health
Hiba Arnaout, Anmol Goel, H. Andrew Schwartz +13
Although artificial intelligence (AI) shows growing promise for mental health care, current approaches to evaluating AI tools in this domain remain fragmented and poorly aligned wi…
The World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities
Harsh Nishant Lalai, Raj Sanjay Shah, Jiaxin Pei +3
Large Language Models (LLMs) have been extensively tuned to mitigate explicit biases, yet they often exhibit subtle implicit biases rooted in their pre-training data. Rather than d…
A Neural Network Model of Complementary Learning Systems: Pattern Separation and Completion for Continual Learning
James P Jun, Vijay Marupudi, Raj Sanjay Shah +1
Learning new information without forgetting prior knowledge is central to human intelligence. In contrast, neural network models suffer from catastrophic forgetting: a significant…
Can LLM-Simulated Practice and Feedback Upskill Human Counselors? A Randomized Study with 90+ Novice Counselors
Ryan Louie, Raj Sanjay Shah, Ifdita Hasan Orney +3
The growing demand for accessible mental health support requires training more counselors, yet existing approaches remain resource-intensive and difficult to scale. LLMs can realis…
TN-Eval: Rubric and Evaluation Protocols for Measuring the Quality of Behavioral Therapy Notes
Raj Sanjay Shah, Lei Xu, Qianchu Liu +3
Behavioral therapy notes are important for both legal compliance and patient care. Unlike progress notes in physical health, quality standards for behavioral therapy notes remain u…
BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop
Lucas Charpentier, Leshem Choshen, Ryan Cotterell +11
BabyLM aims to dissolve the boundaries between cognitive modeling and language modeling. We call for both workshop papers and for researchers to join the 3rd BabyLM competition. As…
The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theories
Raj Sanjay Shah, Sashank Varma
Many studies have evaluated the cognitive alignment of Pre-trained Language Models (PLMs), i.e., their correspondence to adult performance across a range of cognitive domains. Rece…
Understanding Graphical Perception in Data Visualization through Zero-shot Prompting of Vision-Language Models
Grace Guo, Jenna Jiayi Kang, Raj Sanjay Shah +2
Vision Language Models (VLMs) have been successful at many chart comprehension tasks that require attending to both the images of charts and their accompanying textual descriptions…
Development of Cognitive Intelligence in Pre-trained Language Models
Raj Sanjay Shah, Khushi Bhardwaj, Sashank Varma
Recent studies show evidence for emergent cognitive abilities in Large Pre-trained Language Models (PLMs). The increasing cognitive alignment of these models has made them candidat…
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing
Jiangshu Du, Yibo Wang, Wenting Zhao +37
This work is motivated by two key trends. On one hand, large language models (LLMs) have shown remarkable versatility in various generative tasks such as writing, drawing, and ques…
From Intentions to Techniques: A Comprehensive Taxonomy and Challenges in Text Watermarking for Large Language Models
Harsh Nishant Lalai, Aashish Anantha Ramakrishnan, Raj Sanjay Shah +1
With the rapid growth of Large Language Models (LLMs), safeguarding textual content against unauthorized use is crucial. Watermarking offers a vital solution, protecting both - LLM…
How Well Do Deep Learning Models Capture Human Concepts? The Case of the Typicality Effect
Siddhartha K. Vemuri, Raj Sanjay Shah, Sashank Varma
How well do representations learned by ML models align with those of humans? Here, we consider concept representations learned by deep learning models and evaluate whether they sho…
Incremental Comprehension of Garden-Path Sentences by Large Language Models: Semantic Interpretation, Syntactic Re-Analysis, and Attention
Andrew Li, Xianle Feng, Siddhant Narang +4
When reading temporarily ambiguous garden-path sentences, misinterpretations sometimes linger past the point of disambiguation. This phenomenon has traditionally been studied in ps…
Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors
Alicja Chaszczewicz, Raj Sanjay Shah, Ryan Louie +3
Realistic practice and tailored feedback are key processes for training peer counselors with clinical skills. However, existing mechanisms of providing feedback largely rely on hum…
Natural Mitigation of Catastrophic Interference: Continual Learning in Power-Law Learning Environments
Atith Gandhi, Raj Sanjay Shah, Vijay Marupudi +1
Neural networks often suffer from catastrophic interference (CI): performance on previously learned tasks drops off significantly when learning a new task. This contrasts strongly…
What Makes Digital Support Effective? How Therapeutic Skills Affect Clinical Well-Being
Anna Fang, Wenjie Yang, Raj Sanjay Shah +4
Online mental health support communities have grown in recent years for providing accessible mental and emotional health support through volunteer counselors. Despite millions of p…
Pre-training LLMs using human-like development data corpus
Khushi Bhardwaj, Raj Sanjay Shah, Sashank Varma
Pre-trained Large Language Models (LLMs) have shown success in a diverse set of language inference and understanding tasks. The pre-training stage of LLMs looks at a large corpus o…
Human Behavioral Benchmarking: Numeric Magnitude Comparison Effects in Large Language Models
Raj Sanjay Shah, Vijay Marupudi, Reba Koenen +2
Large Language Models (LLMs) do not differentially represent numbers, which are pervasive in text. In contrast, neuroscience research has identified distinct neural representations…