5 papers
Multi-Way Representation Alignment
Akshit Achara, Tatiana Gaintseva, Mateo Mahaut +5
The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces. However, current strategies for mapping t…
Similarity of Processing Steps in Vision Model Representations
Matéo Mahaut, Marco Baroni
Recent literature suggests that the bigger the model, the more likely it is to converge to similar, ``universal'' representations, despite different training objectives, datasets,…
A quantitative analysis of semantic information in deep representations of text and images
Santiago Acevedo, Andrea Mascaretti, Riccardo Rende +3
It was recently observed that the representations of different models that process identical or semantically related inputs tend to align. We analyze this phenomenon using the Info…
Repetitions are not all alike: distinct mechanisms sustain repetition in language models
Matéo Mahaut, Francesca Franzon
Large Language Models (LLMs) can sometimes degrade into repetitive loops, persistently generating identical word sequences. Because repetition is rare in natural human language, it…
Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators
Matéo Mahaut, Laura Aina, Paula Czarnowska +3
Large Language Models (LLMs) tend to be unreliable in the factuality of their answers. To address this problem, NLP researchers have proposed a range of techniques to estimate LLM'…