12 papers
The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs
Ahmed Oumar El-Shangiti, Abzal Nurgazy, Hilal AlQuabeh +2
Despite strong performance on many multimodal tasks, vision-language models (VLMs) still struggle with basic object counting. We investigate whether this reflects missing internal…
Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs
Kareem Elozeiri, Mervat Abassy, Omar Kallas +4
A key challenge in Arabic NLP is the scarcity of dialectal data relative to Modern Standard Arabic (MSA), causing LLMs to overproduce MSA and struggle with dialectally accurate gen…
Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings
Tomomasa Hara, Hiroto Kurita, Masaaki Imaizumi +2
For constructing text embeddings, mean pooling, which averages token embeddings, is the standard approach. This paper examines whether mean pooling actually works well in real mode…
Hidden Failures in Robustness: Why Supervised Uncertainty Quantification Needs Better Evaluation
Joe Stacey, Hadas Orgad, Kentaro Inui +2
Recent work has shown that the hidden states of large language models contain signals useful for uncertainty estimation and hallucination detection, motivating a growing interest i…
WaveSSM: Multiscale State-Space Models for Non-stationary Signal Attention
Ruben Solozabal, Velibor Bojkovic, Hilal Alquabeh +3
State-space models (SSMs) have emerged as a powerful foundation for long-range sequence modeling, with the HiPPO framework showing that continuous-time projection operators can be…
Sycophancy Hides Linearly in the Attention Heads
Rifo Genadi, Munachiso Nwadike, Nurdaulet Mukhituly +3
We find that correct-to-incorrect sycophancy signals are most linearly separable within multi-head attention activations. Motivated by the linear representation hypothesis, we trai…