151 citations · 307 across the 12 of their papers we have counts for
15 papers
Inference and learning in sparse autoencoders as natural gradient flow
Hadi Vafaii, Tejas Rao, David Chanin +6
Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. Th…
Schema-learning and rebinding as mechanisms of in-context learning and emergence
Sivaramakrishnan Swaminathan, Antoine Dedieu, Rajkumar Vasudeva Raju +3
In-context learning (ICL) is one of the most powerful and most unexpected capabilities to emerge in recent transformer-based large language models (LLMs). Yet the mechanisms that u…
Fast exploration and learning of latent graphs with aliased observations
Miguel Lazaro-Gredilla, Ishan Deshpande, Sivaramakrishnan Swaminathan +2
We consider the problem of recovering a latent graph where the observations at each node are \emph{aliased}, and transitions are stochastic. Observations are gathered by an agent t…
PushWorld: A benchmark for manipulation planning with tools and movable obstacles
Ken Kansky, Skanda Vaidyanath, Scott Swingle +3
While recent advances in artificial intelligence have achieved human-level performance in environments like Starcraft and Go, many physical reasoning tasks remain challenging for m…
Learning noisy-OR Bayesian Networks with Max-Product Belief Propagation
Antoine Dedieu, Guangyao Zhou, Dileep George +1
Noisy-OR Bayesian Networks (BNs) are a family of probabilistic graphical models which express rich statistical dependencies in binary data. Variational inference (VI) has been the…
Perturb-and-max-product: Sampling and learning in discrete energy-based models
Miguel Lazaro-Gredilla, Antoine Dedieu, Dileep George
Perturb-and-MAP offers an elegant approach to approximately sample from a energy-based model (EBM) by computing the maximum-a-posteriori (MAP) configuration of a perturbed version…
Learning a generative model for robot control using visual feedback
Nishad Gothoskar, Miguel Lázaro-Gredilla, Abhishek Agarwal +2
We introduce a novel formulation for incorporating visual feedback in controlling robots. We define a generative model from actions to image observations of features on the end-eff…
Learning undirected models via query training
Miguel Lazaro-Gredilla, Wolfgang Lehrach, Dileep George
Typical amortized inference in variational autoencoders is specialized for a single probabilistic query. Here we propose an inference network architecture that generalizes to unsee…
Learning higher-order sequential structure with cloned HMMs
Antoine Dedieu, Nishad Gothoskar, Scott Swingle +3
Variable order sequence modeling is an important problem in artificial and natural intelligence. While overcomplete Hidden Markov Models (HMMs), in theory, have the capacity to rep…
Cortical Microcircuits from a Generative Vision Model
Dileep George, Alexander Lavin, J. Swaroop Guntupalli +3
Understanding the information processing roles of cortical circuits is an outstanding problem in neuroscience and artificial intelligence. The theoretical setting of Bayesian infer…
Variational Rejection Sampling
Aditya Grover, Ramki Gummadi, Miguel Lazaro-Gredilla +2
Learning latent variable models with stochastic variational inference is challenging when the approximate posterior is far from the true posterior, due to high variance in the grad…
Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
Ken Kansky, Tom Silver, David A. Mély +7
The recent adaptation of deep neural network-based methods to reinforcement learning and planning domains has yielded remarkable progress on individual tasks. Nonetheless, progress…
Hierarchical compositional feature learning
Miguel Lázaro-Gredilla, Yi Liu, D. Scott Phoenix +1
We introduce the hierarchical compositional network (HCN), a directed generative model able to discover and disentangle, without supervision, the building blocks of a set of binary…
Gaussian Processes for Nonlinear Signal Processing
Fernando Pérez-Cruz, Steven Van Vaerenbergh, Juan José Murillo-Fuentes +2
Gaussian processes (GPs) are versatile tools that have been successfully employed to solve nonlinear estimation problems in machine learning, but that are rarely used in signal pro…
Overlapping Mixtures of Gaussian Processes for the Data Association Problem
Miguel Lázaro-Gredilla, Steven Van Vaerenbergh, Neil Lawrence
In this work we introduce a mixture of GPs to address the data association problem, i.e. to label a group of observations according to the sources that generated them. Unlike sever…