16 citations · 29 across the 42 of their papers we have counts for
48 papers
SWE-VERIFIER: Selective Verification for Cost-Efficient Coding Agents
Kaijian Zou, Vaibhav Chennareddy, Yunxiang Zhang +5
Coding agents spend substantial computation generating actions and processing long interaction histories. Poor action choices can steer agents into low-value states, wasting comput…
The Culture Within: Towards Mechanistic Understanding of Cultural Reasoning in LLMs
Samee Arif, Furkan Danisman, Angana Borah +6
Large Language Models (LLMs) increasingly operate across culturally diverse settings, yet existing evaluations largely measure cultural behavior at the output level. We introduce C…
LLaDA-PRM: A Bidirectional Step-Level Reasoning Evaluator
Yiming Feng, Naihao Deng, Yulong Chen +1
Step-level reasoning evaluators are commonly based on autoregressive language models, whose causal attention restricts each step representation to the problem, previous steps, and…
One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs
Naihao Deng, Samee Arif, Shuaichen Chang +2
Warning: This submission studies stereotypes and biases, and contains toxic and offensive examples, used for illustration purposes only. Fairness benchmarks such as BBQ have become…
AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLP
Joan Nwatu, Tsedeniya Solomon Amare, Longju Bai +17
Understanding which countries are represented in NLP datasets is essential for identifying gaps, targeting data collection, measuring progress, and informing AI policy. However, ge…
It Takes One to Bias Them All: Breaking Bad with One-Shot GRPO
Naihao Deng, Yilun Zhu, Naichen Shi +2
Warning: This paper contains several toxic and offensive statements. Modern large language models (LLMs) are typically aligned through large-scale post-training to ensure fair and…
Wait, am I Being Fair? Characterizing Deductive Stereotyping and Mitigating It with Fair-GCG
Naihao Deng, Yilun Zhu, Joan Nwatu +2
Warning: This paper contains several toxic and offensive statements. While reasoning generally improves fairness in recent large language models (LLMs), failures persist. In this w…
The Language-Energy Divide: Measuring Energy Costs of Multilingual LLM Inference
Naihao Deng, Alissa Shen, Yiming Feng +5
Large language models (LLMs) are increasingly deployed in multilingual settings, yet the energy costs of serving these models across different languages remain poorly understood. W…
The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
Naihao Deng, Yiming Feng, Chimaobi Okite +4
Warning: This paper studies stereotypes and biases, and contains potentially disturbing examples, used for illustration purposes only. Our findings should not be interpreted as an…
Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models
Angana Borah, Isabelle Augenstein, Rada Mihalcea
Large language models are increasingly used for social decision-making situations that require balancing cultural norms with personal preferences. For example, a user preferring ho…
The Age of Curiosity Meets the Age of AI: Benchmarking Child Safety in Large Language Models
Samee Arif, Angana Borah, Rada Mihalcea
Children increasingly have access to Large Language Models (LLMs), which may expose them to responses that are developmentally inappropriate or require age-sensitive safety, guidan…
How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
Longju Bai, Zhemin Huang, Xingyao Wang +5
The wide adoption of AI agents in complex human workflows is driving rapid growth in LLM token consumption. When agents are deployed on tasks that require a significant amount of t…
SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
Josue Torres-Fonseca, Naihao Deng, Yinpei Dai +5
Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address safety hazards remains insuffic…
DOTRAG: Retrieval-Time Reasoning Along Paths
Larnell Moore, Naihao Deng, Rada Mihalcea +1
Graph Retrieval-Augmented Generation (GraphRAG) is dominated by a retrieve-then-reason paradigm, where context is retrieved using heuristics and then reasoned over. Such methods st…
LUCid: Redefining Relevance For Lifelong Personalization
Chimaobi Okite, Anika Misra, Joyce Chai +1
Work to date has mainly relied on semantic proximity to identify relevant content for lifelong personalization. However, situational relevance is often more important for determini…
One Word at a Time: Incremental Completion Decomposition Breaks LLM Safety
Samee Arif, Naihao Deng, Zhijing Jin +1
Large Language Models (LLMs) are trained to refuse harmful requests, yet they remain vulnerable to jailbreak attacks that exploit weaknesses in conversational safety mechanisms. We…
When Do Language Models Endorse Limitations on Human Rights Principles?
Keenan Samway, Nicole Miu Takagi, Rada Mihalcea +4
As Large Language Models (LLMs) increasingly mediate global information access with the potential to shape public discourse, their alignment with universal human rights principles…
Belief-Sim: Towards Belief-Driven Simulation of Demographic Misinformation Susceptibility
Angana Borah, Zohaib Khan, Rada Mihalcea +1
Misinformation is a growing societal threat, and susceptibility to misinformative claims varies across demographic groups due to differences in underlying beliefs. As Large Languag…
Preserving Historical Truth: Detecting Historical Revisionism in Large Language Models
Francesco Ortu, Joeun Yook, Punya Syon Pandey +5
Large language models (LLMs) are increasingly used as sources of historical information, motivating the need for scalable audits on contested events and politically charged narrati…
Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks
Guangwei Zhang, Jianing Zhu, Cheng Qian +12
We present Copyright Detective, the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs. The system treats copyrigh…
Culture Affordance Atlas: Reconciling Object Diversity Through Functional Mapping
Joan Nwatu, Longju Bai, Oana Ignat +1
Culture shapes the objects people use and for what purposes, yet mainstream Vision-Language (VL) datasets frequently exhibit cultural biases, disproportionately favoring higher-inc…
Are LLMs Good Safety Agents or a Propaganda Engine?
Neemesh Yadav, Francesco Ortu, Jiarui Liu +5
Large Language Models (LLMs) are trained to refuse to respond to harmful content. However, systematic analyses of whether this behavior is truly a reflection of its safety policies…
The Curious Case of Curiosity across Human Cultures and LLMs
Angana Borah, Zhijing Jin, Rada Mihalcea
Recent advances in Large Language Models (LLMs) have expanded their role in human interaction, yet curiosity -- a central driver of inquiry -- remains underexplored in these system…
SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requests
Punya Syon Pandey, Hai Son Le, Devansh Bhardwaj +2
Large language models (LLMs) are increasingly deployed in contexts where their failures can have direct sociopolitical consequences. Yet, existing safety benchmarks rarely test vul…