4 papers · 1 filter
ABOPD: Antibody CDR Design via On-Policy Distillation
Zhuo Yang, Jiaying He, Jiaqing Xie +5
Antibodies are essential therapeutic molecules, and their complementarity-determining regions (CDRs) form the primary antigen-recognition interface. Recent protein generative model…
Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning
Yechen Zhang, Shuhao Xing, Junhao Huang +5
Recent advances in spectral optimization, notably Muon, have demonstrated that constraining update steps to the Stiefel manifold can significantly accelerate training and improve g…
BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning
Yuan Li, Bo Wang, Yufei Gao +4
Proximal constraints are fundamental to the stability of the Large Language Model reinforcement learning. While the canonical clipping mechanism in PPO serves as an efficient surro…
Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
Runyu Peng, Yunhua Zhou, Demin Song +4
In large language models built upon the Transformer architecture, recent studies have shown that inter-head interaction can enhance attention performance. Motivated by this, we pro…