9 citations · 10 across the 8 of their papers we have counts for
5 papers · 1 filter
Position Bias Correction is Insufficient for One-Pass Attention Sorting
Qiong Tang, Xiangkun Hu, Xiangyang Liu +2
Long-context language models suffer from position bias, where information in middle positions is underutilized. Attention Sorting addresses this by iteratively reordering documents…
NLL-Guided Full-Attention Layer Selection for Training-Free Sliding-Window Adaptation
Qiong Tang, Xiangkun Hu, Xiangyang Liu +2
Hybrid attention models that mix full and sliding-window attention across layers offer a promising approach to efficient long-context inference, but the critical question of \emph{…
Output-Space Allocation Costs for Calibration-Guided LLM Compression: An Empirical Study
Qiong Tang, Xiangkun Hu, Xiangyang Liu +2
Training-free compression methods for large language models (LLMs) often use calibration data to guide compression decisions. ROCKET, a recent method combining sparse-dictionary fa…
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
Junxuan Wang, Xuyang Ge, Wentao Shu +4
The hypothesis of Universality in interpretability suggests that different neural networks may converge to implement similar algorithms on similar tasks. In this work, we investiga…
CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors
Peng Li, Tianxiang Sun, Qiong Tang +4
Large language models (LLMs) pre-trained on massive corpora have demonstrated impressive few-shot learning ability on many NLP tasks. A common practice is to recast the task into a…