95 citations · 201 across the 13 of their papers we have counts for
3 papers · 1 filter
Controlling Conditional Language Models without Catastrophic Forgetting
Tomasz Korbak, Hady Elsahar, German Kruszewski +1
Machine learning is shifting towards general-purpose pretrained generative models, trained in a self-supervised manner on large amounts of data, which can then be applied to solve…
Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
Łukasz Kuciński, Tomasz Korbak, Paweł Kołodziej +1
Communication is compositional if complex signals can be represented as a combination of simpler subparts. In this paper, we theoretically show that inductive biases on both the tr…
Energy-Based Models for Code Generation under Compilability Constraints
Tomasz Korbak, Hady Elsahar, Marc Dymetman +1
Neural language models can be successfully trained on source code, leading to applications such as code completion. However, their versatile autoregressive self-supervision objecti…