12 papers
MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression
Guangheng Yang, Zhenliang Ni, Zhenkai Wu +4
Recently, multimodal large-scale reasoning models have demonstrated remarkable capabilities in solving complex tasks through long Chains-of-Thought (M-CoT). However, excessively lo…
MM-IFEval-Pro: A Multilingual and Attack-Resistant Benchmark for Instruction-Following in Vision-Language Models
Changming Xiao, Zhenliang Ni, Jinhui He +2
As vision-language models (VLMs) rapidly advance in image understanding, cross-modal reasoning, and complex instruction execution, instruction-following capability has become a key…
CapProbe: Evaluating Detailed Image Captions via Full-Scene Dense Question Answering
Mouxiao Huang, Qiangyu Yan, Borui Jiang +1
Evaluating detailed image captions from Vision-Language Models (VLMs) requires going beyond surface-level semantic similarity. Reference-based metrics (e.g., CIDEr and SPICE) and L…
SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models
Yaozhi Wen, Jialong Guo, Zhenliang Ni +2
While Vision-Language Models (VLMs) have demonstrated remarkable performance in processing and understanding both text and images, their large parameter sizes lead to significant c…
Thinking-while-speaking: A Controlled, Interleaved Reasoning Method for Real-Time Speech Generation
Xuan Du, Qiangyu Yan, Wenshuo Li +4
The thinking-while-speaking paradigm aims to make AI communication more human. A key challenge is maintaining fluent speech while performing deep reasoning. Our method, InterRS, ta…
TinySAM 2: Extreme Memory Compression for Efficient Track Anything Model
Zhaoyuan Ding, Yijing Yang, Han Shu +1
Segment Anything Model 2 (SAM 2) serves as a core foundation model in the field of video segmentation. Building upon the original SAM model, it introduces a memory bank mechanism a…
SJD-PAC: Accelerating Speculative Jacobi Decoding via Proactive Drafting and Adaptive Continuation
Jialiang Kang, Han Shu, Wenshuo Li +2
Speculative Jacobi Decoding (SJD) offers a draft-model-free approach to accelerate autoregressive text-to-image synthesis. However, the high-entropy nature of visual generation yie…
VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm
Zhenkai Wu, Xiaowen Ma, Zhenliang Ni +4
Vision-language models (VLMs) excel at image understanding tasks, but the large number of visual tokens imposes significant computational costs, hindering deployment on mobile devi…
Ada-MoGE: Adaptive Mixture of Gaussian Expert Model for Time Series Forecasting
Zhenliang Ni, Xiaowen Ma, Zhenkai Wu +3
Multivariate time series forecasts are widely used, such as industrial, transportation and financial forecasts. However, the dominant frequencies in time series may shift with the…
ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decoding
Jialiang Kang, Han Shu, Wenshuo Li +2
Speculative decoding is a widely adopted technique for accelerating inference in large language models (LLMs), yet its application to vision-language models (VLMs) remains underexp…
Expert Merging: Model Merging with Unsupervised Expert Alignment and Importance-Guided Layer Chunking
Dengming Zhang, Xiaowen Ma, Zhenliang Ni +4
Model merging, which combines multiple domain-specialized experts into a single model, offers a practical path to endow Large Language Models (LLMs) and Multimodal Large Language M…
GhostShell: Streaming LLM Function Calls for Concurrent Embodied Programming
Jian Gong, Youwei Huang, Bo Yuan +16
We present GhostShell, a novel approach that leverages Large Language Models (LLMs) for streaming and concurrent behavioral programming in embodied systems. In contrast to predefin…