7 papers
ParallelVLM: Lossless Video-LLM Acceleration with Visual Alignment Aware Parallel Speculative Decoding
Quan Kong, Yuhao Shen, Yicheng Ji +2
Although current Video-LLMs achieve impressive performance in video understanding tasks, their autoregressive decoding efficiency remains constrained by the massive number of video…
VERA: Identifying and Leveraging Visual Evidence Retrieval Heads in Long-Context Understanding
Rongcan Pei, Huan Li, Fang Guo +1
While Vision-Language Models (VLMs) have shown promise in textual understanding, they face significant challenges when handling long context and complex reasoning tasks. In this pa…
Variance-Adaptive Muon: Pre-Orthogonalization Variance Modulation for Efficient Language Model Pretraining
Jingru Li, Yibo Fan, Huan Li
Optimizer design plays a central role in efficient language model pretraining, directly affecting optimization dynamics, convergence speed, and compute cost under fixed training bu…
CTkvr: KV Cache Retrieval for Long-Context LLMs via Centroid then Token Indexing
Kuan Lu, Shuhang Lin, Sai Wu +7
Large language models (LLMs) are increasingly applied in long-context scenarios such as multi-turn conversations. However, long contexts pose significant challenges for inference e…
CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization
Yichen Yan, Ming Zhong, Qi Zhu +3
Multimodal large language models (MLLMs) rely heavily on instruction tuning to align vision and language capabilities, yet the computational cost of training on large-scale dataset…
Not All Data are Good Labels: On the Self-supervised Labeling for Time Series Forecasting
Yuxuan Yang, Dalin Zhang, Yuxuan Liang +3
Time Series Forecasting (TSF) is a crucial task in various domains, yet existing TSF models rely heavily on high-quality data and insufficiently exploit all available data. This pa…
Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting
Siru Zhong, Weilin Ruan, Ming Jin +3
Recent advancements in time series forecasting have explored augmenting models with text or vision modalities to improve accuracy. While text provides contextual understanding, it…