1 citations · 1 across the 5 of their papers we have counts for
5 papers · 1 filter
Kwai Summary Attention V2 Technical Report
OneRec Team, Chenglong Chu, Guohong Mu +30
Summary-token architectures such as Kwai Summary Attention (KSA) compress distant context into learnable Summary token per chunk, and hybrid variants interleave these layers with f…
Unlabeled Data Can Provably Enhance In-Context Learning of Transformers
Renpu Liu, Jing Yang
Large language models (LLMs) exhibit impressive in-context learning (ICL) capabilities, yet the quality of their predictions is fundamentally limited by the few costly labeled demo…
A Shared Low-Rank Adaptation Approach to Personalized RLHF
Renpu Liu, Peng Wang, Donghao Li +2
Reinforcement Learning from Human Feedback (RLHF) has emerged as a pivotal technique for aligning artificial intelligence systems with human values, achieving remarkable success in…
On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery
Renpu Liu, Ruida Zhou, Cong Shen +1
An intriguing property of the Transformer is its ability to perform in-context learning (ICL), where the Transformer can solve different inference tasks without parameter updating…
Federated Representation Learning in the Under-Parameterized Regime
Renpu Liu, Cong Shen, Jing Yang
Federated representation learning (FRL) is a popular personalized federated learning (FL) framework where clients work together to train a common representation while retaining the…