135 citations · 155 across the 6 of their papers we have counts for
6 papers
Deriving Hyperparameter Scaling Laws via Modern Optimization Theory
Egor Shulgin, Dimitri von Rütte, Tianyue H. Zhang +3
Hyperparameter transfer has become an important component of modern large-scale training recipes. Existing methods, such as muP, primarily focus on transfer between model sizes, wi…
Scaling Behavior of Discrete Diffusion Language Models
Dimitri von Rütte, Janis Fluri, Omead Pooladzandi +3
Modern LLM pre-training consumes vast amounts of compute and training data, making the scaling behavior, or scaling laws, of different models a key distinguishing factor. Discrete…
Generalized Interpolating Discrete Diffusion
Dimitri von Rütte, Janis Fluri, Yuhui Ding +3
While state-of-the-art language models achieve impressive results through next-token prediction, they have inherent limitations such as the inability to revise already generated to…
A Language Model's Guide Through Latent Space
Dimitri von Rütte, Sotiris Anagnostidis, Gregor Bachmann +1
Concept guidance has emerged as a cheap and simple way to control the behavior of language models by probing their hidden representations for concept vectors and using them to pert…
OpenAssistant Conversations -- Democratizing Large Language Model Alignment
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte +15
Aligning large language models (LLMs) with human preferences has proven to drastically improve usability and has driven rapid adoption as demonstrated by ChatGPT. Alignment techniq…
FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control
Dimitri von Rütte, Luca Biggio, Yannic Kilcher +1
Generating music with deep neural networks has been an area of active research in recent years. While the quality of generated samples has been steadily increasing, most methods ar…