151 citations · 203 across the 25 of their papers we have counts for
28 papers
Inference and learning in sparse autoencoders as natural gradient flow
Hadi Vafaii, Tejas Rao, David Chanin +6
Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. Th…
Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments
Antoine Dedieu, Wolfgang Lehrach, Guangyao Zhou +2
Despite their stellar performance on a wide range of tasks, including in-context tasks only revealed during inference, vanilla transformers and variants trained for next-token pred…
Schema-learning and rebinding as mechanisms of in-context learning and emergence
Sivaramakrishnan Swaminathan, Antoine Dedieu, Rajkumar Vasudeva Raju +3
In-context learning (ICL) is one of the most powerful and most unexpected capabilities to emerge in recent transformer-based large language models (LLMs). Yet the mechanisms that u…
Neuroscience needs Network Science
Dániel L Barabási, Ginestra Bianconi, Ed Bullmore +17
The brain is a complex system comprising a myriad of interacting elements, posing significant challenges in understanding its structure, function, and dynamics. Network science has…
Fast exploration and learning of latent graphs with aliased observations
Miguel Lazaro-Gredilla, Ishan Deshpande, Sivaramakrishnan Swaminathan +2
We consider the problem of recovering a latent graph where the observations at each node are \emph{aliased}, and transitions are stochastic. Observations are gathered by an agent t…
Graph schemas as abstractions for transfer learning, inference, and planning
J. Swaroop Guntupalli, Rajkumar Vasudeva Raju, Shrinu Kushagra +6
Transferring latent structure from one environment or problem to another is a mechanism by which humans and animals generalize with very little data. Inspired by cognitive and neur…
3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation
Guangyao Zhou, Nishad Gothoskar, Lirui Wang +5
The ability to perceive and understand 3D scenes is crucial for many applications in computer vision and robotics. Inverse graphics is an appealing approach to 3D scene understandi…
PushWorld: A benchmark for manipulation planning with tools and movable obstacles
Ken Kansky, Skanda Vaidyanath, Scott Swingle +3
While recent advances in artificial intelligence have achieved human-level performance in environments like Starcraft and Go, many physical reasoning tasks remain challenging for m…
Learning noisy-OR Bayesian Networks with Max-Product Belief Propagation
Antoine Dedieu, Guangyao Zhou, Dileep George +1
Noisy-OR Bayesian Networks (BNs) are a family of probabilistic graphical models which express rich statistical dependencies in binary data. Variational inference (VI) has been the…
Space is a latent sequence: Structured sequence learning as a unified theory of representation in the hippocampus
Rajkumar Vasudeva Raju, J. Swaroop Guntupalli, Guangyao Zhou +2
Fascinating and puzzling phenomena, such as landmark vector cells, splitter cells, and event-specific representations to name a few, are regularly discovered in the hippocampus. Wi…
DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model
Nishad Gothoskar, Miguel Lázaro-Gredilla, Yasemin Bekiroglu +4
Visual servoing enables robotic systems to perform accurate closed-loop control, which is required in many applications. However, existing methods either require precise calibratio…
PGMax: Factor Graphs for Discrete Probabilistic Graphical Models and Loopy Belief Propagation in JAX
Guangyao Zhou, Antoine Dedieu, Nishanth Kumar +4
PGMax is an open-source Python package for (a) easily specifying discrete Probabilistic Graphical Models (PGMs) as factor graphs; and (b) automatically running efficient and scalab…
Graphical Models with Attention for Context-Specific Independence and an Application to Perceptual Grouping
Guangyao Zhou, Wolfgang Lehrach, Antoine Dedieu +2
Discrete undirected graphical models, also known as Markov Random Fields (MRFs), can flexibly encode probabilistic interactions of multiple variables, and have enjoyed successful a…
Perturb-and-max-product: Sampling and learning in discrete energy-based models
Miguel Lazaro-Gredilla, Antoine Dedieu, Dileep George
Perturb-and-MAP offers an elegant approach to approximately sample from a energy-based model (EBM) by computing the maximum-a-posteriori (MAP) configuration of a perturbed version…
Sample-Efficient L0-L2 Constrained Structure Learning of Sparse Ising Models
Antoine Dedieu, Miguel Lázaro-Gredilla, Dileep George
We consider the problem of learning the underlying graph of a sparse Ising model with nodes from i.i.d. samples. The most recent and best performing approaches combine an e…
From proprioception to long-horizon planning in novel environments: A hierarchical RL model
Nishad Gothoskar, Miguel Lázaro-Gredilla, Dileep George
For an intelligent agent to flexibly and efficiently operate in complex environments, they must be able to reason at multiple levels of temporal, spatial, and conceptual abstractio…
Query Training: Learning a Worse Model to Infer Better Marginals in Undirected Graphical Models with Hidden Variables
Miguel Lázaro-Gredilla, Wolfgang Lehrach, Nishad Gothoskar +3
Probabilistic graphical models (PGMs) provide a compact representation of knowledge that can be queried in a flexible way: after learning the parameters of a graphical model once,…
Learning a generative model for robot control using visual feedback
Nishad Gothoskar, Miguel Lázaro-Gredilla, Abhishek Agarwal +2
We introduce a novel formulation for incorporating visual feedback in controlling robots. We define a generative model from actions to image observations of features on the end-eff…
A Model of Fast Concept Inference with Object-Factorized Cognitive Programs
Daniel P. Sawyer, Miguel Lázaro-Gredilla, Dileep George
The ability of humans to quickly identify general concepts from a handful of images has proven difficult to emulate with robots. Recently, a computer architecture was developed tha…
Learning undirected models via query training
Miguel Lazaro-Gredilla, Wolfgang Lehrach, Dileep George
Typical amortized inference in variational autoencoders is specialized for a single probabilistic query. Here we propose an inference network architecture that generalizes to unsee…
Learning higher-order sequential structure with cloned HMMs
Antoine Dedieu, Nishad Gothoskar, Scott Swingle +3
Variable order sequence modeling is an important problem in artificial and natural intelligence. While overcomplete Hidden Markov Models (HMMs), in theory, have the capacity to rep…
Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs
Miguel Lázaro-Gredilla, Dianhuan Lin, J. Swaroop Guntupalli +1
Humans can infer concepts from image pairs and apply those in the physical world in a completely different setting, enabling tasks like IKEA assembly from diagrams. If robots could…
Cortical Microcircuits from a Generative Vision Model
Dileep George, Alexander Lavin, J. Swaroop Guntupalli +3
Understanding the information processing roles of cortical circuits is an outstanding problem in neuroscience and artificial intelligence. The theoretical setting of Bayesian infer…
Teaching Compositionality to CNNs
Austin Stone, Huayan Wang, Michael Stark +3
Convolutional neural networks (CNNs) have shown great success in computer vision, approaching human-level performance when trained for specific tasks via application-specific loss…