5 papers
A quantitative analysis of semantic information in deep representations of text and images
Santiago Acevedo, Andrea Mascaretti, Riccardo Rende +3
It was recently observed that the representations of different models that process identical or semantically related inputs tend to align. We analyze this phenomenon using the Info…
Differential syntactic and semantic encoding in LLMs
Santiago Acevedo, Alessandro Laio, Marco Baroni
We study how syntactic and semantic information is encoded in inner layer representations of Large Language Models (LLMs), focusing on the very large DeepSeek-V3. We find that, by…
The Effect of Label Noise on the Information Content of Neural Representations
Ali Hussaini Umar, Franky Kevin Nando Tezoh, Jean Barbier +2
In supervised classification tasks, models are trained to predict a label for each data point. In real-world datasets, these labels are often noisy due to annotation errors. While…
Unsupervised detection of semantic correlations in big data
Santiago Acevedo, Alex Rodriguez, Alessandro Laio
In real-world data, information is stored in extremely large feature vectors. These variables are typically correlated due to complex interactions involving many features simultane…
Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations
Lorenzo Basile, Santiago Acevedo, Luca Bortolussi +2
To gain insight into the mechanisms behind machine learning methods, it is crucial to establish connections among the features describing data points. However, these correlations o…