Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Lossless Anti-Distillation Sampling
Zibo Diao, Jingchu Gai, Xinyue Ai +3
Frontier commercial generative models face a growing threat from distillation, whereby a distiller harvests generated responses and trains a competing model of its own at drastical…
cs.LG2024
Do Efficient Transformers Really Save Computation?
Kai Yang, Jan Ackermann, Zhenyu He +6
As transformer-based language models are trained on increasingly large datasets and with vast numbers of parameters, finding more efficient alternatives to the standard Transformer…
cs.LG2024
Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation
Zhenyu He, Guhao Feng, Shengjie Luo +6
In this work, we leverage the intrinsic segmentation of language sequences and design a new positional encoding method called Bilevel Positional Encoding (BiPE). For each position,…