151 citations · 203 across the 25 of their papers we have counts for
8 papers · 1 filter
Inference and learning in sparse autoencoders as natural gradient flow
Hadi Vafaii, Tejas Rao, David Chanin +6
Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. Th…
Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments
Antoine Dedieu, Wolfgang Lehrach, Guangyao Zhou +2
Despite their stellar performance on a wide range of tasks, including in-context tasks only revealed during inference, vanilla transformers and variants trained for next-token pred…
Fast exploration and learning of latent graphs with aliased observations
Miguel Lazaro-Gredilla, Ishan Deshpande, Sivaramakrishnan Swaminathan +2
We consider the problem of recovering a latent graph where the observations at each node are \emph{aliased}, and transitions are stochastic. Observations are gathered by an agent t…
Learning noisy-OR Bayesian Networks with Max-Product Belief Propagation
Antoine Dedieu, Guangyao Zhou, Dileep George +1
Noisy-OR Bayesian Networks (BNs) are a family of probabilistic graphical models which express rich statistical dependencies in binary data. Variational inference (VI) has been the…
PGMax: Factor Graphs for Discrete Probabilistic Graphical Models and Loopy Belief Propagation in JAX
Guangyao Zhou, Antoine Dedieu, Nishanth Kumar +4
PGMax is an open-source Python package for (a) easily specifying discrete Probabilistic Graphical Models (PGMs) as factor graphs; and (b) automatically running efficient and scalab…
Graphical Models with Attention for Context-Specific Independence and an Application to Perceptual Grouping
Guangyao Zhou, Wolfgang Lehrach, Antoine Dedieu +2
Discrete undirected graphical models, also known as Markov Random Fields (MRFs), can flexibly encode probabilistic interactions of multiple variables, and have enjoyed successful a…
Learning undirected models via query training
Miguel Lazaro-Gredilla, Wolfgang Lehrach, Dileep George
Typical amortized inference in variational autoencoders is specialized for a single probabilistic query. Here we propose an inference network architecture that generalizes to unsee…
Hierarchical compositional feature learning
Miguel Lázaro-Gredilla, Yi Liu, D. Scott Phoenix +1
We introduce the hierarchical compositional network (HCN), a directed generative model able to discover and disentangle, without supervision, the building blocks of a set of binary…