284 citations · 1.6k across the 20 of their papers we have counts for
20 papers
SIMA 2: A Generalist Embodied Agent for Virtual Worlds
SIMA team, Adrian Bolton, Alexander Lerchner +63
We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a signifi…
Scaling Instructable Agents Across Many Simulated Worlds
SIMA Team, Maria Abi Raad, Arun Ahuja +91
Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI. Accomplishing this goal requires lear…
SODA: Bottleneck Diffusion Models for Representation Learning
Drew A. Hudson, Daniel Zoran, Mateusz Malinowski +6
We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact rep…
Leveraging VLM-Based Pipelines to Annotate 3D Objects
Rishabh Kabra, Loic Matthey, Alexander Lerchner +1
Pretrained vision language models (VLMs) present an opportunity to caption unlabeled 3D objects at scale. The leading approach to summarize VLM descriptions from different views of…
Combining Behaviors with the Successor Features Keyboard
Wilka Carvalho, Andre Saraiva, Angelos Filos +7
The Option Keyboard (OK) was recently proposed as a method for transferring behavioral knowledge across tasks. OK transfers knowledge by adaptively combining subsets of known behav…
Constellation: Learning relational abstractions over objects for compositional imagination
James C. R. Whittington, Rishabh Kabra, Loic Matthey +2
Learning structured representations of visual scenes is currently a major bottleneck to bridging perception with reasoning. While there has been exciting progress with slot-based m…
SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition
Rishabh Kabra, Daniel Zoran, Goker Erdogan +5
To help agents reason about scenes in terms of their building blocks, we wish to extract the compositional structure of any given scene (in particular, the configuration and charac…
Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents
Jane X. Wang, Michael King, Nicolas Porcel +14
There has been rapidly growing interest in meta-learning as a method for increasing the flexibility and sample efficiency of reinforcement learning. One problem in this area of res…
AlignNet: Unsupervised Entity Alignment
Antonia Creswell, Kyriacos Nikiforou, Oriol Vinyals +8
Recently developed deep learning models are able to learn to segment scenes into component objects without supervision. This opens many new and exciting avenues of research, allowi…
Unsupervised Model Selection for Variational Disentangled Representation Learning
Sunny Duan, Loic Matthey, Andre Saraiva +4
Disentangled representations have recently been shown to improve fairness, data efficiency and generalisation in simple supervised and reinforcement learning tasks. To extend the b…
COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
Nicholas Watters, Loic Matthey, Matko Bosnjak +2
Data efficiency and robustness to task-irrelevant perturbations are long-standing challenges for deep reinforcement learning algorithms. Here we introduce a modular approach to add…
Multi-Object Representation Learning with Iterative Variational Inference
Klaus Greff, Raphaël Lopez Kaufman, Rishabh Kabra +6
Human perception is structured around objects which form the basis for our higher-level cognition and impressive systematic generalization abilities. Yet most work on representatio…
MONet: Unsupervised Scene Decomposition and Representation
Christopher P. Burgess, Loic Matthey, Nicholas Watters +4
The ability to decompose scenes in terms of abstract building blocks is crucial for general intelligence. Where those basic building blocks share meaningful properties, interaction…
Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
Nicholas Watters, Loic Matthey, Christopher P. Burgess +1
We present a simple neural rendering architecture that helps variational autoencoders (VAEs) learn disentangled representations. Instead of the deconvolutional network typically us…
Towards a Definition of Disentangled Representations
Irina Higgins, David Amos, David Pfau +4
How can intelligent agents solve a diverse set of tasks in a data-efficient manner? The disentangled representation learning approach posits that such an agent would benefit from s…
Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
Alessandro Achille, Tom Eccles, Loic Matthey +4
Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propos…
Understanding disentangling in -VAE
Christopher P. Burgess, Irina Higgins, Arka Pal +4
We present new intuitions and theoretical assessments of the emergence of disentangled representation in variational autoencoders. Taking a rate-distortion theory perspective, we s…
DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
Irina Higgins, Arka Pal, Andrei A. Rusu +6
Domain adaptation is an important open problem in deep reinforcement learning (RL). In many scenarios of interest data is hard to obtain, so agents may learn a source policy in a s…
SCAN: Learning Hierarchical Compositional Visual Concepts
Irina Higgins, Nicolas Sonnerat, Loic Matthey +7
The seemingly infinite diversity of the natural world arises from a relatively small set of coherent rules, such as the laws of physics or chemistry. We conjecture that these rules…
Early Visual Concept Learning with Unsupervised Deep Learning
Irina Higgins, Loic Matthey, Xavier Glorot +5
Automated discovery of early visual concepts from raw image data is a major open challenge in AI research. Addressing this problem, we propose an unsupervised approach for learning…