3 citations · 4 across the 4 of their papers we have counts for
6 papers
Aligning Language Models Towards Counterfactual Invariance
Basar Demir, Emily Shen, Ksheeraj Sai Vepuri +5
Small variations in prompt characteristics can meaningfully alter the semantics, tone, and contextual assumptions of large language model outputs. Such characteristics can involve…
MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning
Sourabh Kulkarni, Ksheeraj Sai Vepuri, Basar Demir +8
Reinforcement learning (RL) is widely used in language-model post-training to maximize rewards assigned to individual model outputs, such as scores from binary verifiers or reward…
Evaluating Commercial AI Chatbots as News Intermediaries
Mirac Suzgun, Emily Shen, Federico Bianchi +5
AI chatbots are rapidly shaping how people encounter the news, yet no prior study has systematically measured how accurately these systems, with their proprietary search integratio…
Measuring Agents in Production
Melissa Z. Pan, Negar Arabzadeh, Riccardo Cogo +22
LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments successful. We present the first syst…
Class-RAG: Real-Time Content Moderation with Retrieval Augmented Generation
Jianfa Chen, Emily Shen, Trupti Bavalatti +10
Robust content moderation classifiers are essential for the safety of Generative AI systems. In this task, differences between safe and unsafe inputs are often extremely subtle, ma…
Regret Bounds for Risk-Sensitive Reinforcement Learning
O. Bastani, Y. J. Ma, E. Shen +1
In safety-critical applications of reinforcement learning such as healthcare and robotics, it is often desirable to optimize risk-sensitive objectives that account for tail outcome…