113 citations · 128 across the 10 of their papers we have counts for
3 papers · 1 filter
DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging
Matteo Pagliardini, Amirkeivan Mohtashami, Francois Fleuret +1
The transformer architecture by Vaswani et al. (2017) is now ubiquitous across application domains, from natural language processing to speech processing and image understanding. W…
Controllable Topic-Focused Abstractive Summarization
Seyed Ali Bahrainian, Martin Jaggi, Carsten Eickhoff
Controlled abstractive summarization focuses on producing condensed versions of a source article to cover specific aspects by shifting the distribution of generated text towards a…
Irreducible Curriculum for Language Model Pretraining
Simin Fan, Martin Jaggi
Automatic data selection and curriculum design for training large language models is challenging, with only a few existing methods showing improvements over standard training. Furt…