5 papers · 1 filter
SAGE-OPD: Selective Agent-Guided Intervention for Multi-Turn On-Policy Distillation
Yuhang Zhou, Lizhu Zhang, Yifan Wu +5
On-policy distillation (OPD) improves student models by training them on trajectories induced by their own policy, making it a promising approach for mitigating exposure bias in ag…
Efficient Pre-Training with Token Superposition
Bowen Peng, Théo Gigant, Jeffrey Quesnelle
Pre-training of Large Language Models is often prohibitively expensive and inefficient at scale, requiring complex and invasive modifications in order to achieve high data throughp…
Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulation
Théo Gigant, Bowen Peng, Jeffrey Quesnelle
Subword tokenization is an essential part of modern large language models (LLMs), yet its specific contributions to training efficiency and model performance remain poorly understo…
Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization
Yurui Pan, Ke Xu, Bo Peng
Alignment of large language models (LLMs) via SFT and RLHF/DPO typically ignores the global geometry of the representation space, relying instead on local token likelihoods or scal…
Long Context Pre-Training with Lighthouse Attention
Bowen Peng, Subho Ghosh, Jeffrey Quesnelle
Training causal transformers at extreme sequence lengths is bottlenecked by the quadratic time and memory of scaled dot-product attention (SDPA). In this work, we propose Lighthous…