1 citations · 1 across the 4 of their papers we have counts for
4 papers · 1 filter
LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression
Zishan Shao, Lixun Zhang, Kangning Cui +10
SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison ac…
A Comprehensive Survey on Long Context Language Modeling
Jiaheng Liu, Dawei Zhu, Zhiqi Bai +34
Efficient processing of long contexts has been a persistent pursuit in Natural Language Processing. With the growing number of long documents, dialogues, and other textual data, it…
Long Context Alignment with Short Instructions and Synthesized Positions
Wenhao Wu, Yizhong Wang, Yao Fu +3
Effectively handling instructions with extremely long context remains a challenge for Large Language Models (LLMs), typically necessitating high-quality long data and substantial c…
Retrieval Head Mechanistically Explains Long-Context Factuality
Wenhao Wu, Yizhong Wang, Guangxuan Xiao +2
Despite the recent progress in long-context language models, it remains elusive how transformer-based models exhibit the capability to retrieve relevant information from arbitrary…