6 papers
Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs
Xinming Wang, Weinong Wang, Hongming Yang +13
Hybrid-thinking multimodal large language models (MLLMs) allow a single model to alternate between deliberative thinking and latency-efficient non-thinking inference. Although thes…
LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation
Siqing Song, Chuang Wang, Yong Lang +2
Deploying large language models (LLMs) in resource-constrained environments is hindered by heavy computational and memory requirements. We present LBLLM, a lightweight binarization…
CL-VISTA: Benchmarking Continual Learning in Video Large Language Models
Haiyang Guo, Yichen Shi, Fei Zhu +6
Video Large Language Models (Video-LLMs) require continual learning to adapt to non-stationary real-world data. However, existing benchmarks fall short of evaluating modern foundat…
Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients
Ziwei Xiang, Fanhu Zeng, Hongjian Fang +6
Large Vision Language Models (LVLMs) have achieved remarkable success in a range of downstream tasks that require multimodal interaction, but their capabilities come with substanti…
MCITlib: Multimodal Continual Instruction Tuning Library and Benchmark
Haiyang Guo, Fei Zhu, Hongbo Zhao +5
Continual learning enables AI systems to acquire new knowledge while retaining previously learned information. While traditional unimodal methods have made progress, the rise of Mu…
ModalPrompt: Towards Efficient Multimodal Continual Instruction Tuning with Dual-Modality Guided Prompt
Fanhu Zeng, Fei Zhu, Haiyang Guo +2
Large Multimodal Models (LMMs) exhibit remarkable multi-tasking ability by learning mixed instruction datasets. However, novel tasks would be encountered sequentially in dynamic wo…