5 papers · 1 filter
Sensitivity-Aware Post-Training Quantization for Deep Neural Networks
Zekang Zheng, Haokun Li, Yaofo Chen +2
Model quantization reduces neural network parameter precision to achieve compression, but often compromises accuracy. Existing post-training quantization (PTQ) methods employ itera…
Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization
Shuaicheng Niu, Guohao Chen, Deyu Chen +7
Test-time adaptation (TTA) may fail to improve or even harm the model performance when test data have: 1) mixed distribution shifts, 2) small batch sizes, 3) online imbalanced labe…
Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting
Mingkui Tan, Guohao Chen, Jiaxiang Wu +4
Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Although recent TTA has sh…
Curse of High Dimensionality Issue in Transformer for Long-context Modeling
Shuhai Zhang, Zeng You, Yaofo Chen +5
Transformer-based large language models (LLMs) excel in natural language processing tasks by capturing long-range dependencies through self-attention mechanisms. However, long-cont…
Core Context Aware Transformers for Long Context Language Modeling
Yaofo Chen, Zeng You, Shuhai Zhang +4
Transformer-based Large Language Models (LLMs) have exhibited remarkable success in extensive tasks primarily attributed to self-attention mechanism, which requires a token to cons…