6 papers
On The Effectiveness-Fluency Trade-Off In LLM Conditioning: A Systematic Study
Iuri Macocco, Pau RodrÃguez, Arno Blaas +3
Controlling the output of Large Language Models (LLMs) is a central challenge for their reliable deployment, yet a clear understanding of the involved trade-offs remains elusive. C…
Tracing Computation Density in LLMs
Corentin Kervadec, Iuliia Lysova, Iuri Macocco +2
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs, but it is not clear that they exploit their f…
Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMs
Corentin Kervadec, Iuliia Lysova, Marco Baroni +1
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs. Several studies on LLM efficiency optimizatio…
Not a nuisance but a useful heuristic: Outlier dimensions favor frequent tokens in language models
Iuri Macocco, Nora Graichen, Gemma Boleda +1
We study last-layer outlier dimensions, i.e. dimensions that display extreme activations for the majority of inputs. We show that outlier dimensions arise in many different modern…
Prediction hubs are context-informed frequent tokens in LLMs
Beatrix M. G. Nielsen, Iuri Macocco, Marco Baroni
Hubness, the tendency for a few points to be among the nearest neighbours of a disproportionate number of other points, commonly arises when applying standard distance measures to…
Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM
Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal +2
Contrastive decoding (CD) (Li et al., 2023) improves the next-token distribution of a large expert language model (LM) using a small amateur LM. Although CD is applied to various L…