20 papers
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett +94
Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heter…
"I Strongly Suspect This Website Is a Scam": Benchmarking PII Leakage and Detection without Defense in Autonomous Web Agents
Soham Roy, Sarthakbrata Halder, Arya Bharaty +5
Deceptive web content, widely instantiated across the internet and commonly known as \textit{social-engineering attacks}, manipulates autonomous web agents into submitting users' p…
Auditing Disability Representation in Vision-Language Models
Srikant Panda, Sourabh Singh Yadav, Palkesh Malviya
Vision-language models (VLMs) are increasingly deployed in socially sensitive applications, yet their behavior with respect to disability remains underexplored. We study disability…
DAIQ: Auditing Demographic Attribute Inference from Question in LLMs
Srikant Panda, Hitesh Laxmichand Patel, Shahad Al-Khalifa +3
Recent evaluations of Large language models (LLMs) audit social bias primarily through prompts that explicitly reference demographic attributes, overlooking whether models infer se…
Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
Guijin Son, Donghun Yang, Hitesh Laxmichand Patel +9
Recent frontier models employ long chain-of-thought reasoning to explore solution spaces in context and achieve stonger performance. While many works study distillation to build sm…
World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Models
Eunsu Kim, Junyeong Park, Na Min An +9
In a globalized world, cultural elements from diverse origins frequently appear together within a single visual scene. We refer to these as culture mixing scenarios, yet how Large…