13 citations · 18 across the 15 of their papers we have counts for
6 papers · 1 filter
Optimality of Right-Invariant Priors
Jannis Bolik, Thomas Hofmann
We discuss optimal prediction for families of probability distributions with a locally compact topological group structure. Right-invariant priors were previously shown to yield a…
Local and Global Decoding in Text Generation
Daniel Gareev, Thomas Hofmann, Ezhilmathi Krishnasamy +1
Text generation, a key component in applications such as dialogue systems, relies on decoding algorithms that sample strings from a language model distribution. Traditional methods…
Explicit Word Density Estimation for Language Modelling
Jovan Andonov, Octavian Ganea, Paulina Grnarova +2
Language Modelling has been a central part of Natural Language Processing for a very long time and in the past few years LSTM-based language models have been the go-to method for c…
Causal Estimation of Memorisation Profiles
Pietro Lesci, Clara Meister, Thomas Hofmann +2
Understanding memorisation in language models has practical and societal implications, e.g., studying models' training dynamics or preventing copyright infringements. Prior work de…
The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language Experiments
Anton Schäfer, Shauli Ravfogel, Thomas Hofmann +2
Multilinguality is crucial for extending recent advancements in language modelling to diverse linguistic communities. To maintain high performance while representing multiple langu…
On the Effect of (Near) Duplicate Subwords in Language Modelling
Anton Schäfer, Thomas Hofmann, Imanol Schlag +1
Tokenisation is a core part of language models (LMs). It involves splitting a character sequence into subwords which are assigned arbitrary indices before being served to the LM. W…