24 papers
RefGuard: Identity-Aware Language-Guided Robot Manipulation via Joint Target-Anchor-Frame Grounding
Lan Wei, Kangyi Lu, Yongchen Wang +5
Vision-language-action (VLA) models have substantially advanced language-guided robot manipulation, yet reliable execution still hinges on identifying which physical object an inst…
Bridging Semantics and Physics with Constrained LLMs for Safe and Trustworthy Robotic Manipulation
Wenhao Hong, Lan Wei, Dandan Zhang
A language-guided robot operating in a real kitchen must do more than produce a plan that appears correct. It must also execute that plan safely in cluttered environments under imp…
SoftVTBench: A Deformation-Aware Visuo-Tactile Dataset and Benchmark for Deformable-Object Manipulation
Bowen Jing, Mingxin Wang, Ruiyang Hao +15
Physical interaction quality is central to deformable-object manipulation, yet most benchmarks evaluate task success alone. A policy may complete the task while allowing slip or ca…
SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version)
Bowen Jing, Mingxin Wang, Ruiyang Hao +15
Deformable object manipulation poses challenges beyond task completion: successful execution must also maintain safe physical interaction, holding the object stably without slip or…
TacEvo: Self-Evolving Architecture Discovery for Robotic Tactile Perception via LLM-Driven Quality-Diversity Search
Mohammed AbuSadeh, Lan Wei, Dandan Zhang
Vision-based tactile sensing converts contact-induced surface deformation into images, enabling robots to infer contact forces and fine surface textures that are not accessible thr…
TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception
Lan Wei, Gurmeher Khurana, Sirine Bhouri +6
Vision-based tactile sensors (VBTSs) enable robots to infer contact geometry and force-related cues by imaging deformation through an internal camera, yet generalisation across sen…
Mag-VLA: Vision-Language-Action Model for Bimanual Magnetically Actuated Microrobot Manipulation
Yongchen Wang, Kangyi Lu, Lan Wei +1
Magnetically actuated microrobots have been used as wireless, non-contact manipulation tools at microscales, making them promising for minimally invasive applications. However, the…
A Digital Twin Framework for Virtual Visuo-Haptic Teleoperation of Complex-Shaped Optical Microrobots
Zongcai Tan, Lan Wei, Dandan Zhang
Optical tweezers (OT) provide piconewton-scale manipulation for delicate biomedical tasks, where visuo-haptic feedback can improve operator awareness by conveying interaction-force…
Simultaneous Contact Selection and Planning for Contact-Rich Manipulation with Cascaded Optimization
Zhe Zhang, Xingrong Diao, Haoxiang Liang +4
We propose an optimization-based framework for robust contact-rich manipulation. Recent contact-implicit methods enable online hybrid planning across contact modes, allowing closed…
Dual-Control Frequency-Aware Diffusion Model for Depth-Dependent Optical Microrobot Microscopy Image Generation
Lan Wei, Zongcai Tan, Kangyi Lu +2
Optical microrobots actuated by optical tweezers (OT) are important for cell manipulation and microscale assembly, but their autonomous operation depends on accurate 3D perception.…
Micro-Dexterity in Biological Micromanipulation: Embodiment, Perception, and Control
Kangyi Lu, Lan Wei, Zongcai Tan +1
Microscale manipulation has advanced substantially in controlled locomotion and targeted transport, yet many biomedical applications require precise and adaptive interaction with b…
i-Tac: Inverse Design of 3D-Printed Tactile Elastomers with Scalable and Tunable Optical and Mechanical Properties
Wen Fan, Dandan Zhang
Elastomers are central to vision-based tactile sensors (VBTSs), where they transduce external contact into observable deformation. Different VBTS architectures, however, require di…
Surgi-HDTMR: Closing the Sensorimotor Loop in Bimanual Microsurgery via Haptics, Digital Twin, and Mixed Reality
Songming Ping, Shaoyue Wen, Junhong Chen +3
Robotic microsurgery demands precise bimanual control, intuitive interaction, and informative force feedback. However, most training platforms for robotic microsurgery lack immersi…
DexterousMag: A Reconfigurable Electromagnetic Actuation System for Miniature Helical Robot
Jialin Lin, Dandan Zhang
Despite the promise of magnetically actuated miniature helical robots for minimally invasive interventions, state-of-the-art electromagnetic actuation systems are often space-ineff…
Context-Aware Adaptive Shared Control for Magnetically-Driven Bimanual Dexterous Micromanipulation
Yongchen Wang, Kangyi Lu, Lan Wei +1
Magnetically actuated robots provide a promising untethered platform for navigation in confined environments, enabling biological studies and targeted micro-delivery. However, dext…
MultiDiffSense: Diffusion-Based Multi-Modal Visuo-Tactile Image Generation Conditioned on Object Shape and Contact Pose
Sirine Bhouri, Lan Wei, Jian-Qing Zheng +1
Acquiring aligned visuo-tactile datasets is slow and costly, requiring specialised hardware and large-scale data collection. Synthetic generation is promising, but prior methods ar…
Design and Control of Modular Magnetic Millirobots for Multimodal Locomotion and Shape Reconfiguration
Erik Garcia Oyono, Jialin Lin, Dandan Zhang
Modular small-scale robots offer the potential for on-demand assembly and disassembly, enabling task-specific adaptation in dynamic and constrained environments. However, existing…
TactEx: An Explainable Multimodal Robotic Interaction Framework for Human-Like Touch and Hardness Estimation
Felix Verstraete, Lan Wei, Wen Fan +1
Accurate perception of object hardness is essential for safe and dexterous contact-rich robotic manipulation. Here, we present TactEx, an explainable multimodal robotic interaction…
Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding
Yanzheng Xiang, Lan Wei, Yizhen Yao +8
Parallel diffusion decoding can accelerate diffusion language model inference by unmasking multiple tokens per step, but aggressive parallelism often harms quality. Revocable decod…
SARL: Spatially-Aware Self-Supervised Representation Learning for Visuo-Tactile Perception
Gurmeher Khurana, Lan Wei, Dandan Zhang
Contact-rich robotic manipulation requires representations that encode local geometry. Vision provides global context but lacks direct measurements of properties such as texture an…
Physics-Informed Machine Learning for Efficient Sim-to-Real Data Augmentation in Micro-Object Pose Estimation
Zongcai Tan, Lan Wei, Dandan Zhang
Precise pose estimation of optical microrobots is essential for enabling high-precision object tracking and autonomous biological studies. However, current methods rely heavily on…
Physics-Informed Machine Learning with Adaptive Grids for Optical Microrobot Depth Estimation
Lan Wei, Lou Genoud, Dandan Zhang
Optical microrobots actuated by optical tweezers (OT) offer great potential for biomedical applications such as cell manipulation and microscale assembly. These tasks demand accura…
A Dataset and Benchmarks for Deep Learning-Based Optical Microrobot Pose and Depth Perception
Lan Wei, Dandan Zhang
Optical microrobots, manipulated via optical tweezers (OT), have broad applications in biomedicine. However, reliable pose and depth perception remain fundamental challenges due to…
Coarse-to-Fine Learning for Multi-Pipette Localisation in Robot-Assisted In Vivo Patch-Clamp
Lan Wei, Gema Vera Gonzalez, Phatsimo Kgwarae +4
In vivo image-guided multi-pipette patch-clamp is essential for studying cellular interactions and network dynamics in neuroscience. However, current procedures mainly rely on manu…