8 citations · 23 across the 12 of their papers we have counts for
13 papers
Calibration-Free Surface Normals Estimation in Vision-Based Tactile Sensing using Universal Photometric Stereo
Zdravko Dugonjic, Stefanie Speidel, Roberto Calandra
Vision-based tactile sensors are a popular solution for capturing rich contact surface geometry. However, to obtain high-detail contact surface normals and depth, it is necessary t…
RCT: A Robot-Collected Touch-Vision-Language Dataset for Tactile Generalization
Jingbo He, Michael Färber, Roberto Calandra
For robots manipulating open-world objects, tactile representations must generalize to unseen materials. We introduce RCT (Robotic Contact Tactile), a robot-collected touch-vision-…
SemanticFeels: Semantic Labeling during In-Hand Manipulation
Anas Al Shikh Khalil, Haozhi Qi, Roberto Calandra
As robots become increasingly integrated into everyday tasks, their ability to perceive both the shape and properties of objects during in-hand manipulation becomes critical for ad…
Enhance Vision-based Tactile Sensors via Dynamic Illumination and Image Fusion
Artemii Redkin, Zdravko Dugonjic, Mike Lambeta +1
Vision-based tactile sensors use structured light to measure deformation in their elastomeric interface. Until now, vision-based tactile sensors such as DIGIT and GelSight have bee…
From Simple to Complex Skills: The Case of In-Hand Object Reorientation
Haozhi Qi, Brent Yi, Mike Lambeta +3
Learning policies in simulation and transferring them to the real world has become a promising approach in dexterous manipulation. However, bridging the sim-to-real gap for each ne…
Digitizing Touch with an Artificial Multimodal Fingertip
Mike Lambeta, Tingfan Wu, Ali Sengul +20
Touch is a crucial sensing modality that provides rich information about object properties and interactions with the physical environment. Humans and robots both benefit from using…
Using Fiber Optic Bundles to Miniaturize Vision-Based Tactile Sensors
Julia Di, Zdravko Dugonjic, Will Fu +11
Vision-based tactile sensors have recently become popular due to their combination of low cost, very high spatial resolution, and ease of integration using widely available miniatu…
A Touch, Vision, and Language Dataset for Multimodal Alignment
Letian Fu, Gaurav Datta, Huang Huang +7
Touch is an important sensing modality for humans, but it has not yet been incorporated into a multimodal generative language model. This is partially due to the difficulty of obta…
Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation
Sudharshan Suresh, Haozhi Qi, Tingfan Wu +9
To achieve human-level dexterity, robots must infer spatial awareness from multimodal sensing to reason over contact interactions. During in-hand manipulation of novel objects, suc…
The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback
Nathan Lambert, Roberto Calandra
Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique to make large language models (LLMs) more capable in complex settings. RLHF proceeds as collec…
A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning
Ran Wei, Nathan Lambert, Anthony McDonald +2
Model-based Reinforcement Learning (MBRL) aims to make agents more sample-efficient, adaptive, and explainable by learning an explicit model of the environment. While the capabilit…
General In-Hand Object Rotation with Vision and Touch
Haozhi Qi, Brent Yi, Sudharshan Suresh +4
We introduce RotateIt, a system that enables fingertip-based object rotation along multiple axes by leveraging multimodal sensory inputs. Our system is trained in simulation, where…
Deep Reinforcement Learning for the Joint Control of Traffic Light Signaling and Vehicle Speed Advice
Johannes V. S. Busch, Robert Voelckner, Peter Sossalla +3
Traffic congestion in dense urban centers presents an economical and environmental burden. In recent years, the availability of vehicle-to-anything communication allows for the tra…