7 citations · 16 across the 11 of their papers we have counts for
11 papers
AutoHarness: improving LLM agents by automatically synthesizing a code harness
Xinghua Lou, Miguel Lázaro-Gredilla, Antoine Dedieu +3
Despite significant strides in language models in the last few years, when used as agents, such models often try to perform actions that are not just suboptimal for a given state,…
Diffusion Model Predictive Control
Guangyao Zhou, Sivaramakrishnan Swaminathan, Rajkumar Vasudeva Raju +6
We propose Diffusion Model Predictive Control (D-MPC), a novel MPC approach that learns a multi-step action proposal and a multi-step dynamics model, both using diffusion models, a…
DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors
Joseph Ortiz, Antoine Dedieu, Wolfgang Lehrach +7
Learning from previously collected data via behavioral cloning or offline reinforcement learning (RL) is a powerful recipe for scaling generalist agents by avoiding the need for ex…
What type of inference is planning?
Miguel Lázaro-Gredilla, Li Yang Ku, Kevin P. Murphy +1
Multiple types of inference are available for probabilistic graphical models, e.g., marginal, maximum-a-posteriori, and even marginal maximum-a-posteriori. Which one do researchers…
Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments
Antoine Dedieu, Wolfgang Lehrach, Guangyao Zhou +2
Despite their stellar performance on a wide range of tasks, including in-context tasks only revealed during inference, vanilla transformers and variants trained for next-token pred…
Graph schemas as abstractions for transfer learning, inference, and planning
J. Swaroop Guntupalli, Rajkumar Vasudeva Raju, Shrinu Kushagra +6
Transferring latent structure from one environment or problem to another is a mechanism by which humans and animals generalize with very little data. Inspired by cognitive and neur…
3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation
Guangyao Zhou, Nishad Gothoskar, Lirui Wang +5
The ability to perceive and understand 3D scenes is crucial for many applications in computer vision and robotics. Inverse graphics is an appealing approach to 3D scene understandi…
Space is a latent sequence: Structured sequence learning as a unified theory of representation in the hippocampus
Rajkumar Vasudeva Raju, J. Swaroop Guntupalli, Guangyao Zhou +2
Fascinating and puzzling phenomena, such as landmark vector cells, splitter cells, and event-specific representations to name a few, are regularly discovered in the hippocampus. Wi…
DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model
Nishad Gothoskar, Miguel Lázaro-Gredilla, Yasemin Bekiroglu +4
Visual servoing enables robotic systems to perform accurate closed-loop control, which is required in many applications. However, existing methods either require precise calibratio…
PGMax: Factor Graphs for Discrete Probabilistic Graphical Models and Loopy Belief Propagation in JAX
Guangyao Zhou, Antoine Dedieu, Nishanth Kumar +4
PGMax is an open-source Python package for (a) easily specifying discrete Probabilistic Graphical Models (PGMs) as factor graphs; and (b) automatically running efficient and scalab…
Graphical Models with Attention for Context-Specific Independence and an Application to Perceptual Grouping
Guangyao Zhou, Wolfgang Lehrach, Antoine Dedieu +2
Discrete undirected graphical models, also known as Markov Random Fields (MRFs), can flexibly encode probabilistic interactions of multiple variables, and have enjoyed successful a…