6 papers
How Pragmatics Shape Articulation: A Computational Case Study in STEM ASL Discourse
Saki Imai, Lee Kezar, Laurel Aichler +5
Most state-of-the-art sign language models are trained on interpreter or isolated vocabulary data, which overlooks the variability that characterizes natural dialogue. However, hum…
Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results
Jan Batzner, Sree Harsha Nelaturu, Damian Stachura +45
AI evaluations are widely used for testing and understanding progress. However, the diverse evaluators bring with them inconsistencies that challenge analysis and comparison. First…
MixDPO: Modeling Preference Strength for Pluralistic Alignment
Saki Imai, Pedram Heydari, Anthony Sicilia +3
Preference based alignment objectives implicitly assume that all human preferences are expressed with equal strength. In practice, however, preference strength varies across indivi…
"Nothing about us without us": Perspectives of Global Deaf and Hard-of-hearing Community Members on Sign Language Technologies
Katherine Atwell, Saki Imai, Danielle Bragg +1
There is accelerating interest in sign language technologies (SLTs), with increasing attention from both industry and academia. However, the perspectives of Deaf and Hard-of-hearin…
Measuring How (Not Just Whether) VLMs Build Common Ground
Saki Imai, Mert Ä°nan, Anthony Sicilia +1
Large vision language models (VLMs) increasingly claim reasoning skills, yet current benchmarks evaluate them in single-turn or question answering settings. However, grounding is a…
SiLVERScore: Semantically-Aware Embeddings for Sign Language Generation Evaluation
Saki Imai, Mert Ä°nan, Anthony Sicilia +1
Evaluating sign language generation is often done through back-translation, where generated signs are first recognized back to text and then compared to a reference using text-base…