66 citations · 183 across the 21 of their papers we have counts for
24 papers
A Survey on Self-Improving Test-Time Intelligence: Feedback-Driven Adapting, Learning, and Scaling at Inference
Shuaicheng Niu, Guohao Chen, Yaofo Chen +14
The ability of AI systems to improve their behavior during deployment is becoming increasingly important. As inference moves beyond the static execution of a fixed trained model, a…
LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretraining
Qiuwu Chen, Zimo Liu, Yuchen Li +8
Large language models (LLMs) have achieved remarkable breakthroughs across various applications. However, their architectures remain inefficient in pretraining due to two main limi…
Latent-Condensed Transformer for Efficient Long Context Modeling
Zeng You, Yaofo Chen, Qiuwu Chen +5
Large language models (LLMs) face significant challenges in processing long contexts due to the linear growth of the key-value (KV) cache and quadratic complexity of self-attention…
Precedent-Informed Reasoning: Mitigating Overthinking in Large Reasoning Models via Test-Time Precedent Learning
Qianyue Wang, Jinwu Hu, Huanxiang Lin +5
Reasoning in Large Language Models (LLMs) often suffers from inefficient long chain-of-thought traces with redundant self-exploration and validation, which inflate computational co…
Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Models
Jinwu Hu, Dongjin Yang, Langyu Bian +6
Large language models (LLMs) have demonstrated impressive performance across various language tasks. However, existing LLM reasoning strategies mainly rely on the LLM itself with f…
Beyond Model Scaling: Test-Time Intervention for Efficient Deep Reasoning
Qianyue Wang, Jinwu Hu, Yufeng Wang +5
Large Reasoning Models (LRMs) excel at multi-step reasoning but often suffer from inefficient reasoning processes like overthinking and overshoot, where excessive or misdirected re…
Training-free Context-adaptive Attention for Efficient Long Context Modeling
Zeng You, Yaofo Chen, Shuhai Zhang +5
Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of natural language processing tasks. These capabilities stem primarily from the self-att…
ProCache: Constraint-Aware Feature Caching with Selective Computation for Diffusion Transformer Acceleration
Fanpu Cao, Yaofo Chen, Zeng You +1
Diffusion Transformers (DiTs) have achieved state-of-the-art performance in generative modeling, yet their high computational cost hinders real-time deployment. While feature cachi…
Sensitivity-Aware Post-Training Quantization for Deep Neural Networks
Zekang Zheng, Haokun Li, Yaofo Chen +2
Model quantization reduces neural network parameter precision to achieve compression, but often compromises accuracy. Existing post-training quantization (PTQ) methods employ itera…
Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization
Shuaicheng Niu, Guohao Chen, Deyu Chen +7
Test-time adaptation (TTA) may fail to improve or even harm the model performance when test data have: 1) mixed distribution shifts, 2) small batch sizes, 3) online imbalanced labe…
Curse of High Dimensionality Issue in Transformer for Long-context Modeling
Shuhai Zhang, Zeng You, Yaofo Chen +5
Transformer-based large language models (LLMs) excel in natural language processing tasks by capturing long-range dependencies through self-attention mechanisms. However, long-cont…
Learning to Generate Gradients for Test-Time Adaptation via Test-Time Training Layers
Qi Deng, Shuaicheng Niu, Ronghao Zhang +4
Test-time adaptation (TTA) aims to fine-tune a trained model online using unlabeled testing data to adapt to new environments or out-of-distribution data, demonstrating broad appli…
Towards Long Video Understanding via Fine-detailed Video Story Generation
Zeng You, Zhiquan Wen, Yaofo Chen +4
Long video understanding has become a critical task in computer vision, driving advancements across numerous applications from surveillance to content retrieval. Existing video und…
Core Context Aware Transformers for Long Context Language Modeling
Yaofo Chen, Zeng You, Shuhai Zhang +4
Transformer-based Large Language Models (LLMs) have exhibited remarkable success in extensive tasks primarily attributed to self-attention mechanism, which requires a token to cons…
Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting
Mingkui Tan, Guohao Chen, Jiaxiang Wu +4
Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Although recent TTA has sh…
Towards Robust and Efficient Cloud-Edge Elastic Model Adaptation via Selective Entropy Distillation
Yaofo Chen, Shuaicheng Niu, Yaowei Wang +3
The conventional deep learning paradigm often involves training a deep model on a server and then deploying the model or its distilled ones to resource-limited edge devices. Usuall…
Towards Stable Test-Time Adaptation in Dynamic Wild World
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang +4
Test-time adaptation (TTA) has shown to be effective at tackling distribution shifts between training and testing data by adapting a given model on test samples. However, the onlin…
Pareto-aware Neural Architecture Generation for Diverse Computational Budgets
Yong Guo, Yaofo Chen, Yin Zheng +5
Designing feasible and effective architectures under diverse computational budgets, incurred by different applications/devices, is essential for deploying deep models in real-world…
Automated Dominative Subspace Mining for Efficient Neural Architecture Search
Yaofo Chen, Yong Guo, Daihai Liao +4
Neural Architecture Search (NAS) aims to automatically find effective architectures within a predefined search space. However, the search space is often extremely large. As a resul…
Efficient Test-Time Model Adaptation without Forgetting
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang +4
Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and testing data by adapting a given model w.r.t. any testing sample. This task is particu…
Content-Aware Convolutional Neural Networks
Yong Guo, Yaofo Chen, Mingkui Tan +3
Convolutional Neural Networks (CNNs) have achieved great success due to the powerful feature learning ability of convolution layers. Specifically, the standard convolution traverse…
Contrastive Neural Architecture Search with Neural Architecture Comparators
Yaofo Chen, Yong Guo, Qi Chen +4
One of the key steps in Neural Architecture Search (NAS) is to estimate the performance of candidate architectures. Existing methods either directly use the validation performance…
Pareto-Frontier-aware Neural Architecture Generation for Diverse Budgets
Yong Guo, Yaofo Chen, Yin Zheng +5
Designing feasible and effective architectures under diverse computation budgets incurred by different applications/devices is essential for deploying deep models in practice. Exis…
Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search
Yong Guo, Yaofo Chen, Yin Zheng +4
Neural architecture search (NAS) has become an important approach to automatically find effective architectures. To cover all possible good architectures, we need to search in an e…