4 papers
Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference
Zixuan Lan, Yanhong Li, Jiawei Zhou
Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplications. We propose Reduced Matrix…
SABRE: Scalable and Automated Benchmarking of VLMs under Stress
Zixuan Lan, Luzhe Sun, Matthew R. Walter +1
Vision-language models (VLMs) are improving rapidly, but benchmark development lags behind, making weaknesses hard to identify. Building stress tests is costly: samples must satisf…
Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
Zixuan Lan, Luzhe Sun, Matthew R. Walter +1
Benchmark accuracy is often implicitly assumed to reflect grounded visual understanding in vision-language models (VLMs), yet it remains unclear to what extent such scores truly re…
Text or Pixels? It Takes Half: On the Token Efficiency of Visual Text Inputs in Multimodal LLMs
Yanhong Li, Zixuan Lan, Jiawei Zhou
Large language models (LLMs) and their multimodal variants can now process visual inputs, including images of text. This raises an intriguing question: can we compress textual inpu…