4 papers
Logit Distance Bounds Representational Similarity
Beatrix M. G. Nielsen, Emanuele Marconato, Luigi Gresele +2
For a broad family of discriminative models that includes autoregressive language models, identifiability results imply that if two models induce the same conditional distributions…
What Cosine Similarity of Label Representations Can and Cannot Tell us
Beatrix M. G. Nielsen, Andreas Grivas
Cosine similarity is often used to measure the similarity of vector representations of neural network models. However, the cosine similarity of representations is not guaranteed to…
When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective
Beatrix M. G. Nielsen, Emanuele Marconato, Andrea Dittadi +1
When and why representations learned by different deep neural networks are similar is an active research topic. We choose to address these questions from the perspective of identif…
Prediction hubs are context-informed frequent tokens in LLMs
Beatrix M. G. Nielsen, Iuri Macocco, Marco Baroni
Hubness, the tendency for a few points to be among the nearest neighbours of a disproportionate number of other points, commonly arises when applying standard distance measures to…