37 papers
Bifocal: Training-Free Mixed Granularity Visual Token Pruning for Multimodal LLMs
Peiran Yin, Hao Wu, Zihao Yang +6
Visual token pruning for multimodal large language models (MLLMs) appears highly effective, with existing methods retaining over 95% of unpruned performance on common VQA benchmark…
ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents
Yichu Fang, Sitong Wei, Haozhe Hu +1
Agentic language models repeatedly encode tool and skill schemas that recur across requests in different combinations and orders, preventing standard prefix caching from reusing th…
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Haozhe Hu, Hao Wu, Peiran Yin +3
Pruning is a promising approach for improving the efficiency of large language models (LLMs). Static structured pruning is hardware-friendly, but its input-agnostic computation all…
Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape CoT Calibration
Shuhao Li, Guodong Du, Anhao Zhao +3
Large language models have made strong reasoning gains through supervised fine-tuning, reinforcement learning, and on-policy distillation, yet these post-training methods are usual…
Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models
Chao Han, Haozhe Hu, Xiaoyu Shen
Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation, yet aggressive sparsification can trigger rapid performance d…
From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs
Xin Qiu, Junlong Tong, Yao Zhang +3
Time series analysis has recently been coupled with Large Language Models (LLMs) to leverage their reasoning and world knowledge capabilities, yet gains remain limited. We attribut…
UniRank: Unified Rank Allocation for Low-Rank LLM Compression
Chao Han, Haozhe Hu, Yongjie Du +5
Low-rank decomposition is a promising compression paradigm for large language models (LLMs), yet its effectiveness hinges on rank budget allocation across weight matrices: uniform…
What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis
Xinghao Chen, Chak Tou Leong, Wenjin Guo +3
Latent Chain-of-Thought (CoT) internalizes reasoning within continuous hidden states, offering a promising alternative to verbose discrete reasoning traces. However, robust latent…
ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inference
Yang Tan, Junlong Tong, Linan Yue +3
Streaming VideoLLMs must continuously process incoming video while maintaining low query latency, making both video-ingestion throughput and query-time responsiveness critical for…
miniReranker: Efficient Multimodal Reranking through Visual Cache Reuse and Interaction Sparsity
Yingqi Fan, Xuan Lu, Anhao Zhao +6
Multimodal large language models (MLLMs) have recently shown strong potential as point-wise rerankers by directly modeling query--document relevance through next-token prediction.…
PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation
Anhao Zhao, Junlong Tong, Yingqi Fan +3
Standard on-policy distillation (OPD) for large language models estimates the reverse-KL objective using student-sampled tokens, yielding an unbiased single-sample Monte Carlo esti…
AdaSR: Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization
Junlong Tong, Wenqi Xu, Yingqi Fan +4
Large reasoning models typically follow a read-then-think paradigm: they observe the complete input, reason over a static context, and then produce the answer. Yet many real-world…
CompRank: Efficient LLM Reranking via Token-Level Compression and Decoding-Free Scoring
Xuan Lu, Haohang Huang, Yingqi Fan +5
Large language model (LLM) rerankers have become an important component of modern retrieval and retrieval-augmented generation pipelines, but their high computational cost limits t…
Escaping the KL Agreement Trap in On-Policy Distillation
Haoran Xin, Anhao Zhao, Ying Sun +3
On-policy distillation (OPD) provides dense token-level supervision by asking a teacher to score student-generated rollouts. However, when the student drifts into an unrecoverable…
Momentum for Reasoning: Dense Intrinsic Signals in Policy Optimization
Hao Chen, Zhanming Shen, Liyao Li +8
Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for eliciting long-chain reasoning in large language models. However, existing methods base…
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
Siyang Lyu, Xinghao Chen, Zhijing Sun +3
Chain-of-Thought (CoT) distillation transfers multi-step reasoning from large reasoning models to smaller students, but verbose teacher traces inflate both training and inference c…
Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
Haozhe Hu, Hao Wu, Anhao Zhao +4
Pruning has emerged as a dominant paradigm for accelerating large language model (LLM) inference, spanning a broad spectrum of methods that remove computation across tokens, layers…
When Should Queries Be Decomposed? A Stage-Aware Study of Query Decomposition for Multi-Condition Retrieval
Bochao Yin, Xuan Lu, Zhengyu Qi +1
Multi-condition retrieval requires systems to identify documents that satisfy multiple distinct constraints, moving beyond mere topical relevance. While query decomposition is wide…
ProactiveLLM: Learning Active Interaction for Streaming Large Language Models
Junlong Tong, Yao Zhang, Anhao Zhao +3
Standard Large Language Models (LLMs) follow a read-then-generate paradigm, causing unnecessary latency and computation. Streaming LLMs alleviate this issue by generating while rec…
Adaptive Mass-Segmented KV Compression for Long-Context Reasoning
Junzhe Yang, Xiaoyu Shen
The linear growth of the Key-Value (KV) cache is a critical bottleneck in long-form LLM inference. Existing KV compression methods mitigate this by evicting tokens based on importa…
From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
Hao Chen, Qi Zhang, Liyao Li +7
Adapting Large Language Models (LLMs) to specialized domains typically incurs high data and computational overhead. While prior efficiency efforts have largely treated data selecti…
Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
Anhao Zhao, Haoran Xin, Yingqi Fan +3
Knowledge distillation is central to LLM post-training, yet its design space remains poorly understood, especially alongside reinforcement learning (RL). We show that the prevailin…
Rethinking the Role of LLMs in Time Series Forecasting
Xin Qiu, Junlong Tong, Yirong Sun +3
Large language models (LLMs) have been introduced to time series forecasting (TSF) to incorporate contextual knowledge beyond numerical signals. However, existing studies question…
SkipOPU: An FPGA-based Overlay Processor for Large Language Models with Dynamically Allocated Computation
Zicheng He, Anhao Zhao, Xiaoyu Shen +2
Large language models (LLMs) have achieved remarkable performance across a wide range of tasks, but their inference efficiency remains a critical bottleneck due to rapidly growing…