5 papers · 1 filter
On the Role of Citations in Preference Data
Yu Hou, Hal Daumé, Rachel Rudinger +1
Many NLP tasks require systems to provide attribution in their outputs--i.e. citations to grounding sources. Attribution serves as a bulwark against model hallucination and as a me…
Language Models Predict Empathy Gaps Between Social In-groups and Out-groups
Yu Hou, Hal Daumé, Rachel Rudinger
Studies of human psychology have demonstrated that people are more motivated to extend empathy to in-group members than out-group members (Cikara et al., 2011). In this study, we i…
GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibration
Yoo Yeon Sung, Eve Fleisig, Yu Hou +2
Language models are often miscalibrated, leading to confidently incorrect answers. We introduce GRACE, a benchmark for language model calibration that incorporates comparison with…
Natural Language Inference Improves Compositionality in Vision-Language Models
Paola Cascante-Bonilla, Yu Hou, Yang Trista Cao +2
Compositional reasoning in Vision-Language Models (VLMs) remains challenging as these models often struggle to relate objects, attributes, and spatial relationships. Recent methods…
COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentences
Shikhar Singh, Nuan Wen, Yu Hou +4
Commonsense reasoning is intuitive for humans but has been a long-term challenge for artificial intelligence (AI). Recent advancements in pretrained language models have shown prom…