7 papers · 1 filter
Logit Distance Bounds Representational Similarity
Beatrix M. G. Nielsen, Emanuele Marconato, Luigi Gresele +2
For a broad family of discriminative models that includes autoregressive language models, identifiability results imply that if two models induce the same conditional distributions…
Relational Linear Properties in Language Models: An Empirical Investigation
Giovanni Valer, Luigi Gresele, Marco Bronzini +1
Linear properties are ubiquitous in the representations of language models; however, testing them experimentally remains a challenging task. This work focuses on relational lineari…
Concise and Logically Consistent Conformal Sets for Neuro-Symbolic Concept-Based Models
Samuele Bortolotti, Emanuele Marconato, Andrea Pugnana +2
Neuro-Symbolic Concept-based Models (NeSy-CBMs) are a family of architectures that integrate neural networks with symbolic reasoning for enhanced reliability in high-stakes applica…
Shortcuts and Identifiability in Concept-based Models from a Neuro-Symbolic Lens
Samuele Bortolotti, Emanuele Marconato, Paolo Morettin +2
Concept-based Models are neural networks that learn a concept extractor to map inputs to high-level concepts and an inference layer to translate these into predictions. Ensuring th…
When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective
Beatrix M. G. Nielsen, Emanuele Marconato, Andrea Dittadi +1
When and why representations learned by different deep neural networks are similar is an active research topic. We choose to address these questions from the perspective of identif…
If Concept Bottlenecks are the Question, are Foundation Models the Answer?
Nicola Debole, Pietro Barbiero, Francesco Giannini +3
Concept Bottleneck Models (CBMs) are neural networks designed to conjoin high performance with ante-hoc interpretability. CBMs work by first mapping inputs (e.g., images) to high-l…