6 papers
ConvCue: Complementary Visual Inductive Biases for Vision-Language Models
Zixuan Lan, Shichu Sun
Modern vision-language models (VLMs) achieve strong performance across a broad range of multimodal tasks, yet still struggle with visual questions that require fine-grained discrim…
Distilling Sequential Computation in Transformer Language Models
Zixuan Lan, Jessica Yang, Yanhong Li +2
Transformer language models process sequences token by token in an autoregressive manner, making growing contexts increasingly expensive. Yet many adjacent token spans are highly p…
Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference
Zixuan Lan, Yanhong Li, Jiawei Zhou
Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplications. We propose Reduced Matrix…
SABRE: Scalable and Automated Benchmarking of VLMs under Stress
Zixuan Lan, Luzhe Sun, Matthew R. Walter +1
Vision-language models (VLMs) are improving rapidly, but benchmark development lags behind, making weaknesses hard to identify. Building stress tests is costly: samples must satisf…
Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
Zixuan Lan, Luzhe Sun, Matthew R. Walter +1
Benchmark accuracy is often implicitly assumed to reflect grounded visual understanding in vision-language models (VLMs), yet it remains unclear to what extent such scores truly re…
Text or Pixels? It Takes Half: On the Token Efficiency of Visual Text Inputs in Multimodal LLMs
Yanhong Li, Zixuan Lan, Jiawei Zhou
Large language models (LLMs) and their multimodal variants can now process visual inputs, including images of text. This raises an intriguing question: can we compress textual inpu…