7 citations · 11 across the 12 of their papers we have counts for
7 papers · 1 filter
Group Representational Position Encoding
Yifan Zhang, Zixiang Chen, Yifeng Liu +6
We present GRAPE (Group Representational Position Encoding), a unified framework for positional encoding based on group actions. GRAPE unifies two families of mechanisms: (i) multi…
Robust Layerwise Scaling Rules by Proper Weight Decay Tuning
Zhiyuan Fan, Yifeng Liu, Qingyue Zhao +2
Empirical scaling laws prescribe how to allocate parameters, data, and compute, while maximal-update parameterization (P) enables learning-rate transfer across widths by equaliz…
MARS-M: When Variance Reduction Meets Matrices
Yifeng Liu, Angela Yuan, Quanquan Gu
Matrix-based preconditioned optimizers, such as Muon, have recently been shown to be more efficient than scalar-based optimizers for training large-scale neural networks, including…
A Novel Approach to Differential Privacy with Alpha Divergence
Yifeng Liu, Zehua Wang
As data-driven technologies advance swiftly, maintaining strong privacy measures becomes progressively difficult. Conventional -differential privacy, while prevalent, exhib…
On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning
Yifan Zhang, Yifeng Liu, Huizhuo Yuan +3
Policy gradient algorithms have been successfully applied to enhance the reasoning capabilities of large language models (LLMs). KL regularization is ubiquitous, yet the design sur…
A Survey of Zero-Knowledge Proof Based Verifiable Machine Learning
Zhizhi Peng, Chonghe Zhao, Taotao Wang +7
Machine learning is increasingly deployed through outsourced and cloud-based pipelines, which improve accessibility but also raise concerns about computational integrity, data priv…
Tensor Product Attention Is All You Need
Yifan Zhang, Yifeng Liu, Huizhuo Yuan +4
Scaling language models to handle longer input sequences typically necessitates large key-value (KV) caches, resulting in substantial memory overhead during inference. In this pape…