59 citations · 457 across the 92 of their papers we have counts for
109 papers
Dense Endoluminal Mapping and Localisation from Monocular Bronchoscopy
Salvatore Esposito, Matias Mattamala, Daniel Rebain +4
Bronchoscopy inserts a camera through the airways to diagnose endoluminal disease. Locating the camera along the airway usually requires additional equipment, such as an electromag…
DSM-Free UAV Visual Localization via Latent-Depth Variable Projection and Structure-Adaptive Coupling
Yibin Ye, Xichao Teng, Shuo Chen +4
Visual localization enables unmanned aerial vehicles (UAVs) to maintain absolute positioning in GNSS-denied environments. A common approach matches UAV images with geo-tagged ortho…
ContactKernels: Adaptive Proposals using Geometry-COnditioned Wrenches for Contact-Rich Manipulation
Aditya Kamireddypalli, Matías Mattamala, João Moura +3
Contact-rich insertion requires precise geometric alignment despite uncertainty in perception, geometry, and con- trol. When this uncertainty causes unintended contact, the re- sul…
Decoupled Early Exits for Task-Dependent Compute Allocation in Flow-Matching VLAs
Riccardo Andrea Izzo, Rimvydas Rubavicius, Gianluca Bardaro +3
Flow-matching Vision-Language-Action (VLA) models have emerged as a potential solution for generalist robot control, designed by combining a pretrained Vision-Language Model (VLM)…
Learning to Assemble Novel Structures with Unfamiliar Parts under Semantic Constraints
Jonghyuk Park, Alex Lascarides, Subramanian Ramamoorthy
This paper describes a neurosymbolic architecture for learning to assemble novel structures using evidence from embodied conversations and task demonstrations. We focus on scenario…
Identifying Informative Environments for Cognition Parameter Inference via Bayesian Experimental Design
Manisha Dubey, Rimvydas Rubavicius, N. Siddharth +1
Computational cognitive modeling seeks to infer latent cognitive mechanisms underlying observed behavior. Bayesian inverse planning provides a principled framework for such inferen…
BayesContact: Uncertain Pose Estimation via Visuo-Tactile Proposals and Simulation-based Inference
Aditya Kamireddypalli, Matias Mattamala, Joao Moura +3
Contact-rich manipulation requires pose estimates that are often more accurate than what depth-only sensing provides. Existing methods, relying on vision and contact, employ costly…
RENEW: Towards Learning World Models and Repairing Model Exploitation from Preferences
Logan Mondal Bhamidipaty, Mykel Kochenderfer, Subramanian Ramamoorthy
World models are widely used in offline reinforcement learning (RL) to improve sample efficiency and generate experience beyond a fixed dataset. However, they are vulnerable to mod…
A Minimal Model of Bounded Trade-Off Screening in Multi-Attribute Choice
Manisha Dubey, Anirban Sarkar, Subramanian Ramamoorthy
Human decision-making often involves choosing between multi-attribute alternatives, yet classical models assume fully compensatory utility aggregation despite evidence that people…
Imperfect World Models are Exploitable
Logan Mondal Bhamidipaty, Esmeralda S. Whitammer, David Abel +2
We propose a novel definition of model exploitation in reinforcement learning. Informally, a world model is exploitable if it implies that one policy should be strictly preferred o…
Towards Human Motion World Models via Executable Behaviour Representations
Rimvydas Rubavicius, Manisha Dubey, N. Siddharth +1
Human motion world models should capture motion's intentionality by being executable: adaptable to different actions and capable of assessing motion quality. To achieve this, we in…
Learning to unfold cloth: Scaling up world models to deformable object manipulation
Jack Rome, Stephen James, Subramanian Ramamoorthy
Learning to manipulate cloth is both a paradigmatic problem for robotic research and a problem of immediate relevance to a variety of applications ranging from assistive care to th…
Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulation
Georgios Kamaras, Craig Innes, Subramanian Ramamoorthy
Likelihood-free inference (LFI) enables system identification in complex tasks via black-box modelling, abstracting nonlinearity and stochasticity, and infers a domain distribution…
ROOM: A Physics-Based Continuum Robot Simulator for Photorealistic Medical Datasets Generation
Salvatore Esposito, Matías Mattamala, Daniel Rebain +4
Continuum robots are advancing bronchoscopy procedures by accessing complex lung airways and enabling targeted interventions. However, their development is limited by the lack of r…
Assistax: A Multi-Agent Hardware-Accelerated Reinforcement Learning Benchmark for Assistive Robotics
Leonard Hinckeldey, Elliot Fosong, Rimvydas Rubavicius +6
As embodied autonomous systems capable of assisting humans in daily activities remain a major goal for robotics, efficient and appropriate reinforcement learning (RL) simulation te…
Evaluating Personalized Beneficial Interventions in the Daily Lives of Older Adults Using a Camera
Longfei Chen, Christopher Lochhead, Robert B. Fisher +3
Beneficial daily activity interventions have been shown to improve both the physical and mental health of older adults. However, there is a lack of robust objective metrics and per…
Towards Embodiment Scaling Laws in Robot Locomotion
Bo Ai, Liu Dai, Nico Bohlinger +7
Cross-embodiment generalization underpins the vision of building generalist embodied agents for any robot, yet its enabling factors remain poorly understood. We investigate embodim…
ContactFusion: Stochastic Poisson Surface Maps from Visual and Contact Sensing
Aditya Kamireddypalli, Joao Moura, Russell Buchanan +3
Robust and precise robotic assembly entails insertion of constituent components. Insertion success is hindered when noise in scene understanding exceeds tolerance limits, especiall…
A Distributional Treatment of Real2Sim2Real for Object-Centric Agent Adaptation in Vision-Driven Deformable Linear Object Manipulation
Georgios Kamaras, Subramanian Ramamoorthy
We present an integrated (or end-to-end) framework for the Real2Sim2Real problem of manipulating deformable linear objects (DLOs) based on visual perception. Working with a paramet…
Learning Visually Grounded Domain Ontologies via Embodied Conversation and Explanation
Jonghyuk Park, Alex Lascarides, Subramanian Ramamoorthy
In this paper, we offer a learning framework in which the agent's knowledge gaps are overcome through corrective feedback from a teacher whenever the agent explains its (incorrect)…
Learning a Neural Association Network for Self-supervised Multi-Object Tracking
Shuai Li, Michael Burke, Subramanian Ramamoorthy +1
This paper introduces a novel framework to learn data association for multi-object tracking in a self-supervised manner. Fully-supervised learning methods are known to achieve exce…
Conversational Code Generation: a Case Study of Designing a Dialogue System for Generating Driving Scenarios for Testing Autonomous Vehicles
Rimvydas Rubavicius, Antonio Valerio Miceli-Barone, Alex Lascarides +1
Cyber-physical systems like autonomous vehicles are tested in simulation before deployment, using domain-specific programs for scenario specification. To aid the testing of autonom…
Learning from Demonstration with Implicit Nonlinear Dynamics Models
Peter David Fagan, Subramanian Ramamoorthy
Learning from Demonstration (LfD) is a useful paradigm for training policies that solve tasks involving complex motions, such as those encountered in robotic manipulation. In pract…
SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning
Rimvydas Rubavicius, Peter David Fagan, Alex Lascarides +1
This paper addresses a challenging interactive task learning scenario we call rearrangement under unawareness: an agent must manipulate a rigid-body environment without knowing a k…