Publications (92)
Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks
Jan Schneider, Mridul Mahajan, Le Chen +4
Tendon drives paired with soft muscle actuation enable faster and safer robots while potentially accelerating skill acquisition. Still, these systems are rarely used in practice du…
Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
Adam R. Kosiorek, Hyunjik Kim, Ingmar Posner +1
We present Sequential Attend, Infer, Repeat (SQAIR), an interpretable deep generative model for videos of moving objects. It can reliably discover and track objects throughout the…
Localising Faster: Efficient and precise lidar-based robot localisation in large-scale environments
Li Sun, Daniel Adolfsson, Martin Magnusson +3
This paper proposes a novel approach for global localisation of mobile robots in large-scale environments. Our method leverages learning-based localisation and filtering-based loca…
Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments
Dan Barnes, Will Maddern, Geoffrey Pascoe +1
We present a self-supervised approach to ignoring "distractors" in camera images for the purposes of robustly estimating vehicle motion in cluttered urban environments. We leverage…
Joint Decision-Making in Robot Teleoperation: When are Two Heads Better Than One?
Duc-An Nguyen, Raunak Bhattacharyya, Clara Colombatto +3
Operators working with robots in safety-critical domains have to make decisions under uncertainty, which remains a challenging problem for a single human operator. An open question…
Variational Causal Dynamics: Discovering Modular World Models from Interventions
Anson Lei, Bernhard Schölkopf, Ingmar Posner
Latent world models allow agents to reason about complex environments with high-dimensional observations. However, adapting to new environments and effectively leveraging previous…
APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulation
Yizhe Wu, Oiwi Parker Jones, Martin Engelcke +1
Recent advances in unsupervised learning for object detection, segmentation, and tracking hold significant promise for applications in robotics. A common approach is to frame these…
E(n) Equivariant Normalizing Flows
Victor Garcia Satorras, Emiel Hoogeboom, Fabian B. Fuchs +2
This paper introduces a generative model equivariant to Euclidean symmetries: E(n) Equivariant Normalizing Flows (E-NFs). To construct E-NFs, we take the discriminative E(n) graph…
Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networks
Martin Engelcke, Dushyant Rao, Dominic Zeng Wang +2
This paper proposes a computationally efficient approach to detecting objects natively in 3D point clouds using convolutional neural networks (CNNs). In particular, this is achieve…
The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset
Dan Barnes, Matthew Gadd, Paul Murcutt +2
In this paper we present The Oxford Radar RobotCar Dataset, a new dataset for researching scene understanding using Millimetre-Wave FMCW scanning radar data. The target application…
Reward-Free Curricula for Training Robust World Models
Marc Rigter, Minqi Jiang, Ingmar Posner
There has been a recent surge of interest in developing generally-capable agents that can adapt to new tasks without additional training in the environment. Learning world models f…
Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives
Oliver Groth, Chia-Man Hung, Andrea Vedaldi +1
Visuomotor control (VMC) is an effective means of achieving basic manipulation tasks such as pushing or pick-and-place from raw images. Conditioning VMC on desired goal states is a…
Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
Christian Scherer, Joe Watson, Theo Gruner +3
Distilling expert demonstration data into large generative models using behavioral cloning is a scalable approach to learning capable policies for robotic control, particularly for…
Hierarchical Attentive Recurrent Tracking
Adam R. Kosiorek, Alex Bewley, Ingmar Posner
Class-agnostic object tracking is particularly difficult in cluttered environments as target specific discriminative models cannot be learned a priori. Inspired by how the human vi…
XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning
Daniel Palenicek, Florian Vogt, Joe Watson +2
Sample efficiency is a central property of effective deep reinforcement learning algorithms. Recent work has improved this through added complexity, such as larger models, exotic n…
Intrinsically Interpretable Attention via Sparse Post-Training
Florent Draye, Anson Lei, Hsiao-Ru Pan +2
We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance. Applying a flexible sparsity regularisation under a constrained-…
Kaputt: A Large-Scale Dataset for Visual Defect Detection
Sebastian Höfer, Dorian Henning, Artemij Amiranashvili +6
We present a novel large-scale dataset for defect detection in a logistics setting. Recent work on industrial anomaly detection has primarily focused on manufacturing scenarios wit…
AutoGraph: Predicting Lane Graphs from Traffic Observations
Jannik Zürn, Ingmar Posner, Wolfram Burgard
Lane graph estimation is a long-standing problem in the context of autonomous driving. Previous works aimed at solving this problem by relying on large-scale, hand-annotated lane g…
Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environments
Markus Wulfmeier, Dominic Zeng Wang, Ingmar Posner
In this work, we present an approach to learn cost maps for driving in complex urban environments from a very large number of demonstrations of driving behaviour by human experts.…
Reconstruction Bottlenecks in Object-Centric Generative Models
Martin Engelcke, Oiwi Parker Jones, Ingmar Posner
A range of methods with suitable inductive biases exist to learn interpretable object-centric representations of images without supervision. However, these are largely restricted t…
XQCfD: Accelerating Fast Actor-Critic Algorithms with Prior Data and Prior Policies
Daniel Palenicek, Florian Vogt, Joe Watson +3
For reinforcement learning in the real world online exploration is expensive A common practice in robotic reinforcement learning is to incorporate additional data to improve sample…
World Models via Policy-Guided Trajectory Diffusion
Marc Rigter, Jun Yamada, Ingmar Posner
World models are a powerful tool for developing intelligent agents. By predicting the outcome of a sequence of actions, world models enable policies to be optimised via on-policy r…
ObPose: Leveraging Pose for Object-Centric Scene Inference and Generation in 3D
Yizhe Wu, Oiwi Parker Jones, Ingmar Posner
We present ObPose, an unsupervised object-centric inference and generation model which learns 3D-structured latent representations from RGB-D scenes. Inspired by prior art in 2D re…
Next Steps: Learning a Disentangled Gait Representation for Versatile Quadruped Locomotion
Alexander L. Mitchell, Wolfgang Merkt, Mathieu Geisert +5
Quadruped locomotion is rapidly maturing to a degree where robots now routinely traverse a variety of unstructured terrains. However, while gaits can be varied typically by selecti…
Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks
Peter Ondruska, Ingmar Posner
This paper presents to the best of our knowledge the first end-to-end object tracking approach which directly maps from raw sensor input to object tracks in sensor space without re…
TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfer
Jun Yamada, Marc Rigter, Jack Collins +1
Model-based RL is a promising approach for real-world robotics due to its improved sample efficiency and generalization capabilities compared to model-free RL. However, effective m…
DITTO: Offline Imitation Learning with World Models
Branton DeMoss, Paul Duckworth, Jakob Foerster +2
For imitation learning algorithms to scale to real-world challenges, they must handle high-dimensional observations, offline learning, and policy-induced covariate-shift. We propos…
Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control
Jun Yamada, Adithyavairavan Murali, Ajay Mandlekar +3
Grasping of diverse objects in unstructured environments remains a significant challenge. Open-loop grasping methods, effective in controlled settings, struggle in cluttered enviro…
Neural Latent Geometry Search: Product Manifold Inference via Gromov-Hausdorff-Informed Bayesian Optimization
Haitz Saez de Ocariz Borde, Alvaro Arroyo, Ismael Morales +2
Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather th…
Posterior Sampling Reinforcement Learning with Gaussian Processes for Continuous Control: Sublinear Regret Bounds for Unbounded State Spaces
Hamish Flynn, Joe Watson, Ingmar Posner +1
We analyze the Bayesian regret of the Gaussian process posterior sampling reinforcement learning (GP-PSRL) algorithm. Posterior sampling is a heuristic for decision-making under un…
Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention
Markus W. Baumgartner, Anson Lei, Joe Watson +1
Parametric system identification methods estimate the parameters of explicitly defined physical systems from data. Yet, they remain constrained by the need to provide an explicit f…
Semantically Grounded Object Matching for Robust Robotic Scene Rearrangement
Walter Goodwin, Sagar Vaze, Ioannis Havoutis +1
Object rearrangement has recently emerged as a key competency in robot manipulation, with practical solutions generally involving object detection, recognition, grasping and high-l…
From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery
Ingmar Posner, Anson Lei, Bernhard Schölkopf
The paper introduces Mechanistic World Models, a framework that places reusable explanatory mechanisms at the core of AI systems to enable autonomous scientific discovery beyond me…
Reaching Through Latent Space: From Joint Statistics to Path Planning in Manipulation
Chia-Man Hung, Shaohong Zhong, Walter Goodwin +4
We present a novel approach to path planning for robotic manipulators, in which paths are produced via iterative optimisation in the latent space of a generative model of robot pos…
GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement
Martin Engelcke, Oiwi Parker Jones, Ingmar Posner
Advances in unsupervised learning of object-representations have culminated in the development of a broad range of methods for unsupervised object segmentation and interpretable ob…
Dropout Distillation for Efficiently Estimating Model Confidence
Corina Gurau, Alex Bewley, Ingmar Posner
We propose an efficient way to output better calibrated uncertainty scores from neural networks. The Distilled Dropout Network (DDN) makes standard (non-Bayesian) neural networks m…
Building Gradient by Gradient: Decentralised Energy Functions for Bimanual Robot Assembly
Alexander L. Mitchell, Joe Watson, Ingmar Posner
There are many challenges in bimanual assembly, including high-level sequencing, multi-robot coordination, and low-level, contact-rich operations such as component mating. Task and…
RAMP: A Benchmark for Evaluating Robotic Assembly Manipulation and Planning
Jack Collins, Mark Robson, Jun Yamada +3
We introduce RAMP, an open-source robotics benchmark inspired by real-world industrial assembly tasks. RAMP consists of beams that a robot must assemble into specified goal configu…
Scrutinizing and De-Biasing Intuitive Physics with Neural Stethoscopes
Fabian B. Fuchs, Oliver Groth, Adam R. Kosiorek +4
Visually predicting the stability of block towers is a popular task in the domain of intuitive physics. While previous work focusses on prediction accuracy, a one-dimensional perfo…
Task and Joint Space Dual-Arm Compliant Control
Alexander L. Mitchell, Tobit Flatscher, Ingmar Posner
Robots that interact with humans or perform delicate manipulation tasks must exhibit compliance. However, most commercial manipulators are rigid and suffer from significant frictio…
Probably Unknown: Deep Inverse Sensor Modelling In Radar
Rob Weston, Sarah Cen, Paul Newman +1
Radar presents a promising alternative to lidar and vision in autonomous vehicle applications, able to detect objects at long range under a variety of weather conditions. However,…
End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks
Peter Ondruska, Julie Dequaire, Dominic Zeng Wang +1
In this work we present a novel end-to-end framework for tracking and classifying a robot's surroundings in complex, dynamic and only partially observable real-world environments.…
DreamUp3D: Object-Centric Generative Models for Single-View 3D Scene Understanding and Real-to-Sim Transfer
Yizhe Wu, Haitz Sáez de Ocáriz Borde, Jack Collins +2
3D scene understanding for robotic applications exhibits a unique set of requirements including real-time inference, object-centric latent representation learning, accurate 6D pose…
Universal Approximation of Functions on Sets
Edward Wagstaff, Fabian B. Fuchs, Martin Engelcke +2
Modelling functions of sets, or equivalently, permutation-invariant functions, is a long-standing challenge in machine learning. Deep Sets is a popular method which is known to be…
Find Your Own Way: Weakly-Supervised Segmentation of Path Proposals for Urban Autonomy
Dan Barnes, Will Maddern, Ingmar Posner
We present a weakly-supervised approach to segmenting proposed drivable paths in images with the goal of autonomous driving in complex urban environments. Using recorded routes fro…
Resource-Performance Trade-off Analysis for Mobile Robot Design
Morteza Lahijanian, Maria Svorenova, Akshay A. Morye +6
The design of mobile autonomous robots is challenging due to the limited on-board resources such as processing power and energy. A promising approach is to generate intelligent sch…
The Complexity Dynamics of Grokking
Branton DeMoss, Silvia Sapora, Jakob Foerster +2
We demonstrate the existence of a complexity phase transition in neural networks by studying the grokking phenomenon, where networks suddenly transition from memorization to genera…
LUMOS: Language-Conditioned Imitation Learning with World Models
Iman Nematollahi, Branton DeMoss, Akshay L Chandra +3
We introduce LUMOS, a language-conditioned multi-task imitation learning framework for robotics. LUMOS learns skills by practicing them over many long-horizon rollouts in the laten…
RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces
Sebastien Ehrhardt, Oliver Groth, Aron Monszpart +4
We present RELATE, a model that learns to generate physically plausible scenes and videos of multiple interacting objects. Similar to other generative approaches, RELATE is trained…
Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
Kento Kawaharazuka, Jihoon Oh, Jun Yamada +2
Amid growing efforts to leverage advances in large language models (LLMs) and vision-language models (VLMs) for robotics, Vision-Language-Action (VLA) models have recently gained s…
Efficient Skill Acquisition for Complex Manipulation Tasks in Obstructed Environments
Jun Yamada, Jack Collins, Ingmar Posner
Data efficiency in robotic skill acquisition is crucial for operating robots in varied small-batch assembly settings. To operate in such environments, robots must have robust obsta…
Masking by Moving: Learning Distraction-Free Radar Odometry from Pose Information
Dan Barnes, Rob Weston, Ingmar Posner
This paper presents an end-to-end radar odometry system which delivers robust, real-time pose estimates based on a learned embedding space free of sensing artefacts and distractor…
On the Limitations of Representing Functions on Sets
Edward Wagstaff, Fabian B. Fuchs, Martin Engelcke +2
Recent work on the representation of functions on sets has considered the use of summation in a latent space to enforce permutation invariance. In particular, it has been conjectur…
GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
Martin Engelcke, Adam R. Kosiorek, Oiwi Parker Jones +1
Generative latent-variable models are emerging as promising tools in robotics and reinforcement learning. Yet, even though tasks in these domains typically involve distinct objects…
Is Single-View Mesh Reconstruction Ready for Robotics?
Frederik Nolte, Andreas Geiger, Bernhard Schölkopf +1
This paper evaluates single-view mesh reconstruction models for their potential in enabling instant digital twin creation for real-time planning and dynamics prediction using physi…
Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Models
Tianyou Wang, Anson Lei, Joe Watson +1
Imitation learning has emerged as a powerful paradigm for learning visuomotor policies, but its generalisation and stability are limited by the scale and quality of demonstration d…
Compete and Compose: Learning Independent Mechanisms for Modular World Models
Anson Lei, Frederik Nolte, Bernhard Schölkopf +1
We present COmpetitive Mechanisms for Efficient Transfer (COMET), a modular world model which leverages reusable, independent mechanisms across different environments. COMET is tra…
Introspective Visuomotor Control: Exploiting Uncertainty in Deep Visuomotor Control for Failure Recovery
Chia-Man Hung, Li Sun, Yizhe Wu +2
End-to-end visuomotor control is emerging as a compelling solution for robot manipulation tasks. However, imitation learning-based visuomotor control approaches tend to suffer from…
Addressing Appearance Change in Outdoor Robotics with Adversarial Domain Adaptation
Markus Wulfmeier, Alex Bewley, Ingmar Posner
Appearance changes due to weather and seasonal conditions represent a strong impediment to the robust implementation of machine learning systems in outdoor robotics. While supervis…
Deep Tracking on the Move: Learning to Track the World from a Moving Vehicle using Recurrent Neural Networks
Julie Dequaire, Dushyant Rao, Peter Ondruska +2
This paper presents an end-to-end approach for tracking static and dynamic objects for an autonomous vehicle driving through crowded urban environments. Unlike traditional approach…
Fast-MbyM: Leveraging Translational Invariance of the Fourier Transform for Efficient and Accurate Radar Odometry
Robert Weston, Matthew Gadd, Daniele De Martini +2
Masking By Moving (MByM), provides robust and accurate radar odometry measurements through an exhaustive correlative search across discretised pose candidates. However, this dense…
D-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous Deformable Manipulation
Jun Yamada, Shaohong Zhong, Jack Collins +1
Mastering dexterous robotic manipulation of deformable objects is vital for overcoming the limitations of parallel grippers in real-world applications. Current trajectory optimisat…
You Only Look at One: Category-Level Object Representations for Pose Estimation From a Single Example
Walter Goodwin, Ioannis Havoutis, Ingmar Posner
In order to meaningfully interact with the world, robot manipulators must be able to interpret objects they encounter. A critical aspect of this interpretation is pose estimation:…
Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar
Dan Barnes, Ingmar Posner
This paper presents a self-supervised framework for learning to detect robust keypoints for odometry estimation and metric localisation in radar. By embedding a differentiable poin…
SPARTAN: A Sparse Transformer World Model Attending to What Matters
Anson Lei, Bernhard Schölkopf, Ingmar Posner
Capturing the interactions between entities in a structured way plays a central role in world models that flexibly adapt to changes in the environment. Recent works motivate the be…
VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representation
Alexander L. Mitchell, Wolfgang Merkt, Mathieu Geisert +5
Quadruped locomotion is rapidly maturing to a degree where robots are able to realise highly dynamic manoeuvres. However, current planners are unable to vary key gait parameters of…
Modelling Observation Correlations for Active Exploration and Robust Object Detection
Javier Velez, Garrett Hemann, Albert S. Huang +2
Today, mobile robots are expected to carry out increasingly complex tasks in multifarious, real-world environments. Often, the tasks require a certain semantic understanding of the…
End-to-end Recurrent Multi-Object Tracking and Trajectory Prediction with Relational Reasoning
Fabian B. Fuchs, Adam R. Kosiorek, Li Sun +2
The majority of contemporary object-tracking approaches do not model interactions between objects. This contrasts with the fact that objects' paths are not independent: a cyclist m…
TACO: Learning Task Decomposition via Temporal Alignment for Control
Kyriacos Shiarlis, Markus Wulfmeier, Sasha Salter +2
Many advanced Learning from Demonstration (LfD) methods consider the decomposition of complex, real-world tasks into simpler sub-tasks. By reusing the corresponding sub-policies wi…
Projections of Model Spaces for Latent Graph Inference
Haitz Sáez de Ocáriz Borde, Ãlvaro Arroyo, Ingmar Posner
Graph Neural Networks leverage the connectivity structure of graphs as an inductive bias. Latent graph inference focuses on learning an adequate graph structure to diffuse informat…
From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
Nicholas Roy, Ingmar Posner, Tim Barfoot +17
Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains. Consequently, the notion of applying learning metho…
Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learning
Sasha Salter, Kristian Hartikainen, Walter Goodwin +1
The ability to discover behaviours from past experience and transfer them to new tasks is a hallmark of intelligent agents acting sample-efficiently in the real world. Equipping em…
Iterative SE(3)-Transformers
Fabian B. Fuchs, Edward Wagstaff, Justas Dauparas +1
When manipulating three-dimensional data, it is possible to ensure that rotational and translational symmetries are respected by applying so-called SE(3)-equivariant models. Protei…
Attention-Privileged Reinforcement Learning
Sasha Salter, Dushyant Rao, Markus Wulfmeier +2
Image-based Reinforcement Learning is known to suffer from poor sample efficiency and generalisation to unseen visuals such as distractors (task-independent aspects of the observat…
What Makes a Place? Building Bespoke Place Dependent Object Detectors for Robotics
Jeffrey Hawke, Alex Bewley, Ingmar Posner
This paper is about enabling robots to improve their perceptual performance through repeated use in their operating environment, creating local expert detectors fitted to the place…
Incorporating Human Domain Knowledge into Large Scale Cost Function Learning
Markus Wulfmeier, Dushyant Rao, Ingmar Posner
Recent advances have shown the capability of Fully Convolutional Neural Networks (FCN) to model cost functions for motion planning in the context of learning driving preferences pu…
Maximum Entropy Deep Inverse Reinforcement Learning
Markus Wulfmeier, Peter Ondruska, Ingmar Posner
This paper presents a general framework for exploiting the representational capacity of neural networks to approximate complex, nonlinear reward functions in the context of solving…
COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping
Jun Yamada, Alexander L. Mitchell, Jack Collins +1
This paper addresses the challenge of occluded robot grasping, i.e. grasping in situations where the desired grasp poses are kinematically infeasible due to environmental constrain…
Zero-Shot Category-Level Object Pose Estimation
Walter Goodwin, Sagar Vaze, Ioannis Havoutis +1
Object pose estimation is an important component of most vision pipelines for embodied agents, as well as in 3D vision more generally. In this paper we tackle the problem of estima…
No Epoch Like the Present: Robust Climate Emulation Requires Out-of-Distribution Generalisation
Bradley Stanley-Clamp, Anson Lei, Hannah M. Christensen +1
Climate emulation is an out-of-distribution (OOD) projection task. This is precisely the challenge where modern Machine Learning (ML) methods are most prone to failure. Consequentl…
ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking
Oliver Groth, Fabian B. Fuchs, Ingmar Posner +1
Physical intuition is pivotal for intelligent agents to perform complex tasks. In this paper we investigate the passive acquisition of an intuitive understanding of physical princi…
Gaitor: Learning a Unified Representation Across Gaits for Real-World Quadruped Locomotion
Alexander L. Mitchell, Wolfgang Merkt, Aristotelis Papatheodorou +2
The current state-of-the-art in quadruped locomotion is able to produce a variety of complex motions. These methods either rely on switching between a discrete set of skills or lea…
Offline Adaptation of Quadruped Locomotion using Diffusion Models
Reece O'Mahoney, Alexander L. Mitchell, Wanming Yu +2
We present a diffusion-based approach to quadrupedal locomotion that simultaneously addresses the limitations of learning and interpolating between multiple skills and of (modes) o…
Gromov-Hausdorff Distances for Comparing Product Manifolds of Model Spaces
Haitz Saez de Ocariz Borde, Alvaro Arroyo, Ismael Morales +2
Recent studies propose enhancing machine learning models by aligning the geometric characteristics of the latent space with the underlying data structure. Instead of relying solely…
Incremental Adversarial Domain Adaptation for Continually Changing Environments
Markus Wulfmeier, Alex Bewley, Ingmar Posner
Continuous appearance shifts such as changes in weather and lighting conditions can impact the performance of deployed machine learning models. While unsupervised domain adaptation…
Mutual Alignment Transfer Learning
Markus Wulfmeier, Ingmar Posner, Pieter Abbeel
Training robots for operation in the real world is a complex, time consuming and potentially expensive task. Despite significant success of reinforcement learning in games and simu…
Leveraging Scene Embeddings for Gradient-Based Motion Planning in Latent Space
Jun Yamada, Chia-Man Hung, Jack Collins +2
Motion planning framed as optimisation in structured latent spaces has recently emerged as competitive with traditional methods in terms of planning success while significantly out…
Imagine That! Leveraging Emergent Affordances for 3D Tool Synthesis
Yizhe Wu, Sudhanshu Kasewa, Oliver Groth +4
In this paper we explore the richness of information captured by the latent space of a vision-based generative model. The model combines unsupervised generative learning with a tas…
Enhancing Joint Human-AI Inference in Robot Missions: A Confidence-Based Approach
Duc-An Nguyen, Clara Colombatto, Steve Fleming +3
Joint human-AI inference holds immense potential to improve outcomes in human-supervised robot missions. Current day missions are generally in the AI-assisted setting, where the hu…
First Steps: Latent-Space Control with Semantic Constraints for Quadruped Locomotion
Alexander L. Mitchell, Martin Engelcke, Oiwi Parker Jones +5
Traditional approaches to quadruped control frequently employ simplified, hand-derived models. This significantly reduces the capability of the robot since its effective kinematic…
A Review of Differentiable Simulators
Rhys Newbury, Jack Collins, Kerry He +4
Differentiable simulators continue to push the state of the art across a range of domains including computational physics, robotics, and machine learning. Their main value is the a…
There and Back Again: Learning to Simulate Radar Data for Real-World Applications
Rob Weston, Oiwi Parker Jones, Ingmar Posner
Simulating realistic radar data has the potential to significantly accelerate the development of data-driven approaches to radar processing. However, it is fraught with difficulty…