2 papers
cs.CL2024
Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models
Zhisong Zhang, Yan Wang, Xinting Huang +5
Large language models have shown remarkable performance across a wide range of language tasks, owing to their exceptional capabilities in context modeling. The most commonly used m…
cs.CL2024
On the Transformations across Reward Model, Parameter Update, and In-Context Prompt
Deng Cai, Huayang Li, Tingchen Fu +11
Despite the general capabilities of pre-trained large language models (LLMs), they still need further adaptation to better serve practical applications. In this paper, we demonstra…