1 citations · 1 across the 5 of their papers we have counts for
6 papers
Does Reasoning Improve Psychological Depth in Large Language Models? It Depends on Who's Judging
Ruichen Zheng, Yihe Wang, Fabrice Y Harel-Canada +4
LLM-as-a-Judge evaluators are increasingly used to score open-ended generation, yet a judge's correlation with human ratings on its development set may not guarantee valid measurem…
NormViz: A Benchmark and Framework for Grounding Multimodal Reasoning in Global Cultures
Akhila Yerukola, Fabrice Y Harel-Canada, Simran Khanuja +5
AI systems are used worldwide, but they struggle to serve the needs of culturally diverse populations. Prior work on cultural understanding evaluates AI systems on text-only settin…
VaPR -- Vision-language Preference alignment for Reasoning
Rohan Wadhawan, Fabrice Y Harel-Canada, Zi-Yi Dou +3
Preference finetuning methods like Direct Preference Optimization (DPO) with AI-generated feedback have shown promise in aligning Large Vision-Language Models (LVLMs) with human pr…
Sandcastles in the Storm: Revisiting the (Im)possibility of Strong Watermarking
Fabrice Y Harel-Canada, Boran Erol, Connor Choi +4
Watermarking AI-generated text is critical for combating misuse. Yet recent theoretical work argues that any watermark can be erased via random walk attacks that perturb text while…
Measuring Psychological Depth in Language Models
Fabrice Harel-Canada, Hanyu Zhou, Sreya Muppalla +4
Evaluations of creative stories generated by large language models (LLMs) often focus on objective properties of the text, such as its style, coherence, and diversity. While these…
EnDex: Evaluation of Dialogue Engagingness at Scale
Guangxuan Xu, Ruibo Liu, Fabrice Harel-Canada +2
We propose EnDex, the first human-reaction based model to evaluate dialogue engagingness. EnDex is trained on 80k Reddit-based Engagement Dataset (RED) curated using a novel distan…