4 papers · 1 filter
End-to-end Planner Training for Language Modeling
Nathan Cornille, Florian Mai, Jingyuan Sun +1
Through end-to-end training to predict the next token, LLMs have become valuable tools for various tasks. Enhancing their core training in language modeling can improve numerous do…
Learning to Plan Long-Term for Language Modeling
Florian Mai, Nathan Cornille, Marie-Francine Moens
Modern language models predict the next token in the sequence by considering the past text through a powerful function such as attention. However, language models have no explicit…
Learning to Plan for Language Modeling from Unlabeled Data
Nathan Cornille, Marie-Francine Moens, Florian Mai
By training to predict the next token in an unlabeled corpus, large language models learn to perform many tasks without any labeled data. However, their next-token-prediction objec…
Triple-Encoders: Representations That Fire Together, Wire Together
Justus-Jonas Erker, Florian Mai, Nils Reimers +2
Search-based dialog models typically re-encode the dialog history at every turn, incurring high cost. Curved Contrastive Learning, a representation learning method that encodes rel…