Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Improving Instruct Models for Free: A Study on Partial Adaptation
Ozan Ä°rsoy, Pengxiang Cheng, Jennifer L. Chen +3
Instruct models, obtained from various instruction tuning or post-training steps, are commonly deemed superior and more usable than their base counterpart. While the model gains in…
cs.CL2024
MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies
Shiyue Zhang, Shijie Wu, Ozan Irsoy +4
Autoregressive language models are trained by minimizing the cross-entropy of the model distribution Q relative to the data distribution P -- that is, minimizing the forward cross-…