14 papers
Anti-Shortcut Distillation via Temporal Negative Knowledge Transfer
Syed Muhammad Raza, Omer Tariq, Jeongbae Son
Knowledge distillation (KD) trains a compact student by attracting it towards a converged teacher. It is silent about which directions the teacher itself learned to suppress: repul…
Latent-Condensed Transformer for Efficient Long Context Modeling
Zeng You, Yaofo Chen, Qiuwu Chen +5
Large language models (LLMs) face significant challenges in processing long contexts due to the linear growth of the key-value (KV) cache and quadratic complexity of self-attention…
Precedent-Informed Reasoning: Mitigating Overthinking in Large Reasoning Models via Test-Time Precedent Learning
Qianyue Wang, Jinwu Hu, Huanxiang Lin +5
Reasoning in Large Language Models (LLMs) often suffers from inefficient long chain-of-thought traces with redundant self-exploration and validation, which inflate computational co…
ProCache: Constraint-Aware Feature Caching with Selective Computation for Diffusion Transformer Acceleration
Fanpu Cao, Yaofo Chen, Zeng You +1
Diffusion Transformers (DiTs) have achieved state-of-the-art performance in generative modeling, yet their high computational cost hinders real-time deployment. While feature cachi…
Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Models
Jinwu Hu, Dongjin Yang, Langyu Bian +6
Large language models (LLMs) have demonstrated impressive performance across various language tasks. However, existing LLM reasoning strategies mainly rely on the LLM itself with f…
Beyond Model Scaling: Test-Time Intervention for Efficient Deep Reasoning
Qianyue Wang, Jinwu Hu, Yufeng Wang +5
Large Reasoning Models (LRMs) excel at multi-step reasoning but often suffer from inefficient reasoning processes like overthinking and overshoot, where excessive or misdirected re…