5 papers · 1 filter
LRHP: Learning Representations for Human Preferences via Preference Pairs
Chenglong Wang, Yang Gan, Yifu Huo +7
To improve human-preference alignment training, current research has developed numerous preference datasets consisting of preference pairs labeled as "preferred" or "dispreferred".…
NDP: Next Distribution Prediction as a More Broad Target
Junhao Ruan, Abudukeyumu Abudula, Xinyu Liu +7
Large language models (LLMs) trained on next-token prediction (NTP) paradigm have demonstrated powerful capabilities. However, the existing NTP paradigm contains several limitation…
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking
Chi Hu, Chenglong Wang, Xiangnan Ma +5
This paper addresses the efficiency challenge of Neural Architecture Search (NAS) by formulating the task as a ranking problem. Previous methods require numerous training examples…
The NiuTrans System for WNGT 2020 Efficiency Task
Chi Hu, Bei Li, Ye Lin +5
This paper describes the submissions of the NiuTrans Team to the WNGT 2020 Efficiency Shared Task. We focus on the efficient implementation of deep Transformer models \cite{wang-et…
The NiuTrans System for the WMT21 Efficiency Task
Chenglong Wang, Chi Hu, Yongyu Mu +8
This paper describes the NiuTrans system for the WMT21 translation efficiency task (http://statmt.org/wmt21/efficiency-task.html). Following last year's work, we explore various te…