5 papers
State Beyond Appearance: Diagnosing and Improving State Consistency in Dial-Based Measurement Reading
Yuanze Hu, Gen Li, Yuqin Lan +5
Multimodal large language models (MLLMs) have achieved impressive progress on general multimodal tasks, yet they remain brittle on dial-based measurement reading. In this paper, we…
Mosaic: Multimodal Jailbreak against Closed-Source VLMs via Multi-View Ensemble Optimization
Yuqin Lan, Gen Li, Yuanze Hu +6
Vision-Language Models (VLMs) are powerful but remain vulnerable to multimodal jailbreak attacks. Existing attacks mainly rely on either explicit visual prompt attacks or gradient-…
Lyapunov Probes for Hallucination Detection in Large Foundation Models
Bozhi Luan, Gen Li, Yalan Qin +6
We address hallucination detection in Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) by framing the problem through the lens of dynamical systems stabili…
Can Structured Templates Facilitate LLMs in Tackling Harder Tasks? : An Exploration of Scaling Laws by Difficulty
Zhichao Yang, Zhaoxin Fan, Gen Li +6
Structured, procedural reasoning is essential for Large Language Models (LLMs), especially in mathematics. While post-training methods have improved LLM performance, they still fal…
TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottlenecks
Yuanze Hu, Zhaoxin Fan, Xinyu Wang +8
Lightweight Vision-Language Models (VLMs) are indispensable for resource-constrained applications. The prevailing approach to aligning vision and language models involves freezing…