Publications (92)
OVGrasp: Open-Vocabulary Grasping Assistance via Multimodal Intent Detection
Chen Hu, Shan Luo, Letizia Gionfrida
Grasping assistance is essential for restoring autonomy in individuals with motor impairments, particularly in unstructured environments where object categories and user intentions…
An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
Chao Xu, Suyu Zhang, Yang Liu +11
Vision-Language-Action (VLA) models are driving a revolution in robotics, enabling machines to understand instructions and interact with the physical world. This field is exploding…
TacEva: A Performance Evaluation Framework For Vision-Based Tactile Sensors
Qingzheng Cong, Steven Oh, Wen Fan +3
Vision-Based Tactile Sensors (VBTSs) are widely used in robotic tasks because of the high spatial resolution they offer and their relatively low manufacturing costs. However, varia…
Deep Domain Adaptation Regression for Force Calibration of Optical Tactile Sensors
Zhuo Chen, Ni Ou, Jiaqi Jiang +1
Optical tactile sensors provide robots with rich force information for robot grasping in unstructured environments. The fast and accurate calibration of three-dimensional contact f…
TopoFR: A Closer Look at Topology Alignment on Face Recognition
Jun Dan, Yang Liu, Jiankang Deng +4
The field of face recognition (FR) has undergone significant advancements with the rise of deep learning. Recently, the success of unsupervised learning and graph neural networks h…
Fully Convolutional One-Shot Object Segmentation for Industrial Robotics
Benjamin Schnieders, Shan Luo, Gregory Palmer +1
The ability to identify and localize new objects robustly and effectively is vital for robotic grasping and manipulation in warehouses or smart factories. Deep convolutional neural…
In-Device Feedback in Immersive Head-Mounted Displays for Distance Perception During Teleoperation of Unmanned Ground Vehicles
Yiming Luo, Jialin Wang, Rongkai Shi +2
In recent years, Virtual Reality (VR) Head-Mounted Displays (HMD) have been used to provide an immersive, first-person view in real-time for the remote-control of Unmanned Ground V…
STS MICCAI 2023 Challenge: Grand challenge on 2D and 3D semi-supervised tooth segmentation
Yaqi Wang, Yifan Zhang, Xiaodiao Chen +24
Computer-aided design (CAD) tools are increasingly popular in modern dental practice, particularly for treatment planning or comprehensive prognosis evaluation. In particular, the…
ViTacTip: Design and Verification of a Novel Biomimetic Physical Vision-Tactile Fusion Sensor
Wen Fan, Haoran Li, Weiyong Si +3
Tactile sensing is significant for robotics since it can obtain physical contact information during manipulation. To capture multimodal contact information within a compact framewo…
Generation of GelSight Tactile Images for Sim2Real Learning
Daniel Fernandes Gomes, Paolo Paoletti, Shan Luo
Most current works in Sim2Real learning for robotic manipulation tasks leverage camera vision that may be significantly occluded by robot hands during the manipulation. Tactile sen…
SimTac: A Physics-Based Simulator for Vision-Based Tactile Sensing with Biomorphic Structures
Xuyang Zhang, Jiaqi Jiang, Zhuo Chen +5
Tactile sensing in biological organisms is deeply intertwined with morphological form, such as human fingers, cat paws, and elephant trunks, which enables rich and adaptive interac…
Knock-Knock: Acoustic Object Recognition by using Stacked Denoising Autoencoders
Shan Luo, Leqi Zhu, Kaspar Althoefer +1
This paper presents a successful application of deep learning for object recognition based on acoustic data. The shortcomings of previously employed approaches where handcrafted fe…
ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI
Stone Tao, Fanbo Xiang, Arth Shukla +20
Simulation has enabled unprecedented compute-scalable approaches to robot learning. However, many existing simulation frameworks typically support a narrow range of scenes/tasks an…
End-to-End Dexterous Grasp Learning from Single-View Point Clouds via a Multi-Object Scene Dataset
Tao Geng, Dapeng Yang, Ziwei Liu +6
Dexterous grasping in multi-object scene constitutes a fundamental challenge in robotic manipulation. Current mainstream grasping datasets predominantly focus on single-object scen…
Training Tactile Sensors to Learn Force Sensing from Each Other
Zhuo Chen, Ni Ou, Xuyang Zhang +8
Humans achieve stable and dexterous object manipulation by coordinating grasp forces across multiple fingers and palms, facilitated by a unified tactile memory system in the somato…
Robotic Tactile Perception of Object Properties: A Review
Shan Luo, Joao Bimbo, Ravinder Dahiya +1
Touch sensing can help robots understand their sur- rounding environment, and in particular the objects they interact with. To this end, roboticists have, in the last few decades,…
Adaptive Parametric Activation: Unifying and Generalising Activation Functions Across Tasks
Konstantinos Panagiotis Alexandridis, Jiankang Deng, Anh Nguyen +1
The activation function plays a crucial role in model optimisation, yet the optimal choice remains unclear. For example, the Sigmoid activation is the de-facto activation in balanc…
Surface Following using Deep Reinforcement Learning and a GelSightTactile Sensor
Chen Lu, Jing Wang, Shan Luo
Tactile sensors can provide detailed contact in-formation that can facilitate robots to perform dexterous, in-hand manipulation tasks. One of the primitive but important tasks is s…
ViTac: Feature Sharing between Vision and Tactile Sensing for Cloth Texture Recognition
Shan Luo, Wenzhen Yuan, Edward Adelson +2
Vision and touch are two of the important sensing modalities for humans and they offer complementary information for sensing the environment. Robots could also benefit from such mu…
Road Surface Defect Detection -- From Image-based to Non-image-based: A Survey
Jongmin Yu, Jiaqi Jiang, Sebastiano Fichera +4
Ensuring traffic safety is crucial, which necessitates the detection and prevention of road surface defects. As a result, there has been a growing interest in the literature on the…
TransFace++: Rethinking the Face Recognition Paradigm with a Focus on Accuracy, Efficiency, and Security
Jun Dan, Yang Liu, Baigui Sun +2
Face Recognition (FR) technology has made significant strides with the emergence of deep learning. Typically, most existing FR models are built upon Convolutional Neural Networks (…
RoTip: A Finger-Shaped Tactile Sensor with Active Rotation Capability
Xuyang Zhang, Jiaqi Jiang, Shan Luo
In recent years, advancements in optical tactile sensor technology have primarily centred on enhancing sensing precision and expanding the range of sensing modalities. To meet the…
Iterative Closest Labeled Point for Tactile Object Shape Recognition
Shan Luo, Wenxuan Mou, Kaspar Althoefer +1
Tactile data and kinesthetic cues are two important sensing sources in robot object recognition and are complementary to each other. In this paper, we propose a novel algorithm nam…
iCLAP: Shape Recognition by Combining Proprioception and Touch Sensing
Shan Luo, Wenxuan Mou, Kaspar Althoefer +1
For humans, both the proprioception and touch sensing are highly utilized when performing haptic perception. However, most approaches in robotics use only either proprioceptive dat…
ConViTac: Aligning Visual-Tactile Fusion with Contrastive Representations
Zhiyuan Wu, Yongqiang Zhao, Shan Luo
Vision and touch are two fundamental sensory modalities for robots, offering complementary information that enhances perception and manipulation tasks. Previous research has attemp…
Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors
Zixi Chen, Shixin Zhang, Shan Luo +2
Simulation is widely applied in robotics research to save time and resources. There have been several works to simulate optical tactile sensors that leverage either a smoothing met…
Marker or Markerless? Mode-Switchable Optical Tactile Sensing for Diverse Robot Tasks
Ni Ou, Zhuo Chen, Shan Luo
Optical tactile sensors play a pivotal role in robot perception and manipulation tasks. The membrane of these sensors can be painted with markers or remain markerless, enabling the…
VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds
Zhiyuan Wu, Zhuo Chen, Shan Luo
Vision and touch are complementary modalities essential for robotic perception and manipulation. While vision provides global object context, touch offers precise local information…
Fly Safe: Aerial Swarm Robotics using Force Field Particle Swarm Optimisation
Lauren Parker, James Butterworth, Shan Luo
Particle Swarm Optimisation (PSO) is a powerful optimisation algorithm that can be used to locate global maxima in a search space. Recent interest in swarms of Micro Aerial Vehicle…
Extended BIC for linear regression models with diverging number of relevant features and high or ultra-high feature spaces
Shan Luo, Zehua Chen
In many conventional scientific investigations with high or ultra-high dimensional feature spaces, the relevant features, though sparse, are large in number compared with classical…
Monoscopic vs. Stereoscopic Views and Display Types in the Teleoperation of Unmanned Ground Vehicles for Object Avoidance
Yiming Luo, Jialin Wang, Hai-Ning Liang +2
Virtual reality (VR) head-mounted displays (HMD) have recently been used to provide an immersive, first-person vision/view in real-time for manipulating remotely-controlled unmanne…
UniForce: A Unified Latent Force Model for Robot Manipulation with Diverse Tactile Sensors
Zhuo Chen, Fei Ni, Kaiyao Luo +7
Force sensing is essential for dexterous robot manipulation, but scaling force-aware policy learning is hindered by the heterogeneity of tactile sensors. Differences in sensing pri…
Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping
Jiaqi Jiang, Guanqun Cao, Aaron Butterworth +2
Picking up transparent objects is still a challenging task for robots. The visual properties of transparent objects such as reflection and refraction make the current grasping meth…
Multimodal perception for dexterous manipulation
Guanqun Cao, Shan Luo
Humans usually perceive the world in a multimodal way that vision, touch, sound are utilised to understand surroundings from various dimensions. These senses are combined together…
BER Analysis of SCMA-OFDM Systems in the Presence of Carrier Frequency Offset
Haibo Liu, Qu Luo, Zilong Liu +3
Sparse code multiple access (SCMA) building upon orthogonal frequency division multiplexing (OFDM) is a promising wireless technology for supporting massive connectivity in future…
Inverse Image Frequency for Long-tailed Image Recognition
Konstantinos Panagiotis Alexandridis, Shan Luo, Anh Nguyen +2
The long-tailed distribution is a common phenomenon in the real world. Extracted large scale image datasets inevitably demonstrate the long-tailed property and models trained with…
GelTip Tactile Sensor for Dexterous Manipulation in Clutter
Daniel Fernandes Gomes, Shan Luo
Tactile sensing is an essential capability for robots that carry out dexterous manipulation tasks. While cameras, Lidars and other remote sensors can assess a scene globally and in…
TC-SfM: Robust Track-Community-Based Structure-from-Motion
Lei Wang, Linlin Ge, Shan Luo +3
Structure-from-Motion (SfM) aims to recover 3D scene structures and camera poses based on the correspondences between input images, and thus the ambiguity caused by duplicate struc…
TransForce: Transferable Force Prediction for Vision-based Tactile Sensors with Sequential Image Translation
Zhuo Chen, Ni Ou, Xuyang Zhang +1
Vision-based tactile sensors (VBTSs) provide high-resolution tactile images crucial for robot in-hand manipulation. However, force sensing in VBTSs is underutilized due to the cost…
Spatio-temporal Attention Model for Tactile Texture Recognition
Guanqun Cao, Yi Zhou, Danushka Bollegala +1
Recently, tactile sensing has attracted great interest in robotics, especially for facilitating exploration of unstructured environments and effective manipulation. A detailed unde…
Embodied Arena: A Comprehensive, Unified, and Evolving Evaluation Platform for Embodied AI
Fei Ni, Min Zhang, Pengyi Li +34
Embodied AI development significantly lags behind large foundation models due to three critical challenges: (1) lack of systematic understanding of core capabilities needed for Emb…
When Vision Meets Touch: A Contemporary Review for Visuotactile Sensors from the Signal Processing Perspective
Shoujie Li, Zihan Wang, Changsheng Wu +6
Tactile sensors, which provide information about the physical properties of objects, are an essential component of robotic systems. The visuotactile sensing technology with the mer…
CDI3D: Cross-guided Dense-view Interpolation for 3D Reconstruction
Zhiyuan Wu, Xibin Song, Senbo Wang +8
3D object reconstruction from single-view image is a fundamental task in computer vision with wide-ranging applications. Recent advancements in Large Reconstruction Models (LRMs) h…
Beyond Flat GelSight Sensors: Simulation of Optical Tactile Sensors of Complex Morphologies for Sim2Real Learning
Daniel Fernandes Gomes, Paolo Paoletti, Shan Luo
Recently, several morphologies, each with its advantages, have been proposed for the \textit{GelSight} high-resolution tactile sensors. However, existing simulation methods are lim…
Design and Benchmarking of A Multi-Modality Sensor for Robotic Manipulation with GAN-Based Cross-Modality Interpretation
Dandan Zhang, Wen Fan, Jialin Lin +5
In this paper, we present the design and benchmark of an innovative sensor, ViTacTip, which fulfills the demand for advanced multi-modal sensing in a compact design. A notable feat…
TouchRoller: A Rolling Optical Tactile Sensor for Rapid Assessment of Large Surfaces
Guanqun Cao, Jiaqi Jiang, Chen Lu +2
Tactile sensing is important for robots to perceive the world as it captures the texture and hardness of the object in contact and is robust to illumination and colour variances. H…
Soft Contact Simulation and Manipulation Learning of Deformable Objects with Vision-based Tactile Sensor
Jianhua Shan, Yuhao Sun, Shixin Zhang +7
Deformable object manipulation is a classical and challenging research area in robotics. Compared with rigid object manipulation, this problem is more complex due to the deformatio…
Visual-Tactile Multimodality for Following Deformable Linear Objects Using Reinforcement Learning
Leszek Pecyna, Siyuan Dong, Shan Luo
Manipulation of deformable objects is a challenging task for a robot. It will be problematic to use a single sensory input to track the behaviour of such objects: vision can be sub…
Fast Composite Optimization and Statistical Recovery in Federated Learning
Yajie Bao, Michael Crawshaw, Shan Luo +1
As a prevalent distributed learning paradigm, Federated Learning (FL) trains a global model on a massive amount of devices with infrequent communication. This paper investigates a…
A4T: Hierarchical Affordance Detection for Transparent Objects Depth Reconstruction and Manipulation
Jiaqi Jiang, Guanqun Cao, Thanh-Toan Do +1
Transparent objects are widely used in our daily lives and therefore robots need to be able to handle them. However, transparent objects suffer from light reflection and refraction…
ViTac-Tracing: Visual-Tactile Imitation Learning of Deformable Object Tracing
Yongqiang Zhao, Haining Luo, Yupeng Wang +3
Deformable objects often appear in unstructured configurations. Tracing deformable objects helps bringing them into extended states and facilitating the downstream manipulation tas…
GarmNet: Improving Global with Local Perception for Robotic Laundry Folding
Daniel Fernandes Gomes, Shan Luo, Luis F. Teixeira
Developing autonomous assistants to help with domestic tasks is a vital topic in robotics research. Among these tasks, garment folding is one of them that is still far from being a…
Visual-Tactile Peg-in-Hole Assembly Learning from Peg-out-of-Hole Disassembly
Yongqiang Zhao, Xuyang Zhang, Zhuo Chen +3
Peg-in-hole (PiH) assembly is a fundamental yet challenging robotic manipulation task. While reinforcement learning (RL) has shown promise in tackling such tasks, it requires exten…
Learn from Incomplete Tactile Data: Tactile Representation Learning with Masked Autoencoders
Guanqun Cao, Jiaqi Jiang, Danushka Bollegala +1
The missing signal caused by the objects being occluded or an unstable sensor is a common challenge during data collection. Such missing signals will adversely affect the results o…
Representation and Processing of Instantaneous and Durative Temporal Phenomena
Manolis Pitsikalis, Alexei Lisitsa, Shan Luo
Event definitions in Complex Event Processing systems are constrained by the expressiveness of each system's language. Some systems allow the definition of instantaneous complex ev…
Long-tailed Instance Segmentation using Gumbel Optimized Loss
Konstantinos Panagiotis Alexandridis, Jiankang Deng, Anh Nguyen +1
Major advancements have been made in the field of object detection and segmentation recently. However, when it comes to rare categories, the state-of-the-art methods fail to detect…
UniMorphGrasp: Diffusion Model with Morphology-Awareness for Cross-Embodiment Dexterous Grasp Generation
Zhiyuan Wu, Xiangyu Zhang, Zhuo Chen +3
Cross-embodiment dexterous grasping aims to generate stable and diverse grasps for robotic hands with heterogeneous kinematic structures. Existing methods are often tailored to spe…
PointGrasp: Point Cloud-based Grasping for Tendon-driven Soft Robotic Glove Applications
Chen Hu, Shirui Lyu, Eojin Rho +3
Controlling hand exoskeletons to assist individuals with grasping tasks poses a challenge due to the difficulty in understanding user intentions. We propose that most daily graspin…
Logic Rules Meet Deep Learning: A Novel Approach for Ship Type Classification
Manolis Pitsikalis, Thanh-Toan Do, Alexei Lisitsa +1
The shipping industry is an important component of the global trade and economy, however in order to ensure law compliance and safety it needs to be monitored. In this paper, we pr…
Leveraging Multi-modal Sensing for Robotic Insertion Tasks in R&D Laboratories
Aaron Butterworth, Gabriella Pizzuto, Leszek Pecyna +2
Performing a large volume of experiments in Chemistry labs creates repetitive actions costing researchers time, automating these routines is highly desirable. Previous experiments…
ManiSkill-ViTac 2025: Challenge on Manipulation Skill Learning With Vision and Tactile Sensing
Chuanyu Li, Renjun Dang, Xiang Li +8
This article introduces the ManiSkill-ViTac Challenge 2025, which focuses on learning contact-rich manipulation skills using both tactile and visual sensing. Expanding upon the 202…
Vision-Guided Active Tactile Perception for Crack Detection and Reconstruction
Jiaqi Jiang, Guanqun Cao, Daniel Fernandes Gomes +1
Crack detection is of great significance for monitoring the integrity and well-being of the infrastructure such as bridges and underground pipelines, which are harsh environments f…
Multi-class Road Defect Detection and Segmentation using Spatial and Channel-wise Attention for Autonomous Road Repairing
Jongmin Yu, Chen Bene Chi, Sebastiano Fichera +3
Road pavement detection and segmentation are critical for developing autonomous road repair systems. However, developing an instance segmentation method that simultaneously perform…
Localizing the Object Contact through Matching Tactile Features with Visual Map
Shan Luo, Wenxuan Mou, Kaspar Althoefer +1
This paper presents a novel framework for integration of vision and tactile sensing by localizing tactile readings in a visual object map. Intuitively, there are some correspondenc…
MLLM-Fabric: Multimodal Large Language Model-Driven Robotic Framework for Fabric Sorting and Selection
Liman Wang, Hanyang Zhong, Tianyuan Wang +2
Choosing appropriate fabrics is critical for meeting functional and quality demands in robotic textile manufacturing, apparel production, and smart retail. We propose MLLM-Fabric,…
Multimodal Zero-Shot Learning for Tactile Texture Recognition
Guanqun Cao, Jiaqi Jiang, Danushka Bollegala +2
Tactile sensing plays an irreplaceable role in robotic material recognition. It enables robots to distinguish material properties such as their local geometry and textures, especia…
Adversarially Domain-adaptive Latent Diffusion for Unsupervised Semantic Segmentation
Jongmin Yu, Zhongtian Sun, Chen Bene Chi +2
Semantic segmentation requires extensive pixel-level annotation, motivating unsupervised domain adaptation (UDA) to transfer knowledge from labelled source domains to unlabelled or…
GelTip: A Finger-shaped Optical Tactile Sensor for Robotic Manipulation
Daniel Fernandes Gomes, Zhonglin Lin, Shan Luo
Sensing contacts throughout the fingers is an essential capability for a robot to perform manipulation tasks in cluttered environments. However, existing tactile sensors either onl…
Vis2Hap: Vision-based Haptic Rendering by Cross-modal Generation
Guanqun Cao, Jiaqi Jiang, Ningtao Mao +3
To assist robots in teleoperation tasks, haptic rendering which allows human operators access a virtual touch feeling has been developed in recent years. Most previous haptic rende…
Phase Noise Resilient Codebook Design for Sparse Code Multiple Access
Haibo Liu, Qu Luo, Zilong Liu +3
Sparse code multiple access (SCMA) is a promising technique for future machine type communication systems due to its superior spectral efficiency and capability for supporting mass…
Selection Consistency of EBIC for GLIM with Non-canonical Links and Diverging Number of Parameters
Shan Luo, Zehua Chen
In this article, we investigate the properties of the EBIC in variable selection for generalized linear models with non-canonical links and diverging number of parameters in ultra-…
A Case Study on Visual-Audio-Tactile Cross-Modal Retrieval
Jagoda Wojcik, Jiaqi Jiang, Jiacheng Wu +1
Cross-Modal Retrieval (CMR), which retrieves relevant items from one modality (e.g., audio) given a query in another modality (e.g., visual), has undergone significant advancements…
Neural Logic Reinforcement Learning
Zhengyao Jiang, Shan Luo
Deep reinforcement learning (DRL) has achieved significant breakthroughs in various tasks. However, most DRL algorithms suffer a problem of generalizing the learned policy which ma…
HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning
Amirhosein Alian, Yongqiang Zhao, Shiyi Gu +5
Despite the importance of tactile sensing for reliable manipulation, most existing Vision-Language-Action (VLA) datasets remain vision-only, and those that do incorporate tactile i…
"Touching to See" and "Seeing to Feel": Robotic Cross-modal SensoryData Generation for Visual-Tactile Perception
Jet-Tsyn Lee, Danushka Bollegala, Shan Luo
The integration of visual-tactile stimulus is common while humans performing daily tasks. In contrast, using unimodal visual or tactile perception limits the perceivable dimensiona…
Tactile Robotics: An Outlook
Shan Luo, Nathan F. Lepora, Wenzhen Yuan +3
Robotics research has long sought to give robots the ability to perceive the physical world through touch in an analogous manner to many biological systems. Developing such tactile…
RoTipBot: Robotic Handling of Thin and Flexible Objects using Rotatable Tactile Sensors
Jiaqi Jiang, Xuyang Zhang, Daniel Fernandes Gomes +2
This paper introduces RoTipBot, a novel robotic system for handling thin, flexible objects. Different from previous works that are limited to singulating them using suction cups or…
An Efficient Dynamic Multi-Sources To Single-Destination (DMS-SD) Algorithm In Smart City Navigation Using Adjacent Matrix
Ziren Xiao, Ruxin Xiao, Chang Liu +4
Dijkstra's algorithm is one of the most popular classic path planning algorithms, achieving optimal solutions across a wide range of challenging tasks. However, it only calculates…
Robotic Perception of Object Properties using Tactile Sensing
Jiaqi Jiang, Shan Luo
The sense of touch plays a key role in enabling humans to understand and interact with surrounding environments. For robots, tactile sensing is also irreplaceable. While interactin…
Fractal Calibration for long-tailed object detection
Konstantinos Panagiotis Alexandridis, Ismail Elezi, Jiankang Deng +2
Real-world datasets follow an imbalanced distribution, which poses significant challenges in rare-category object detection. Recent studies tackle this problem by developing re-wei…
Point Cloud-based Grasping for Soft Hand Exoskeleton
Chen Hu, Enrica Tricomi, Eojin Rho +4
Grasping is a fundamental skill for interacting with and manipulating objects in the environment. However, this ability can be challenging for individuals with hand impairments. So…
A Message Passing Detection based Affine Frequency Division Multiplexing Communication System
Lifan Wu, Shan Luo, Dongxiao Song +2
The next generation of wireless communication technology is anticipated to address the communication reliability challenges encountered in high-speed mobile communication scenarios…
Reducing Tactile Sim2Real Domain Gaps via Deep Texture Generation Networks
Tudor Jianu, Daniel Fernandes Gomes, Shan Luo
Recently simulation methods have been developed for optical tactile sensors to enable the Sim2Real learning, i.e., firstly training models in simulation before deploying them on th…
FOTS: A Fast Optical Tactile Simulator for Sim2Real Learning of Tactile-motor Robot Manipulation Skills
Yongqiang Zhao, Kun Qian, Boyi Duan +1
Simulation is a widely used tool in robotics to reduce hardware consumption and gather large-scale data. Despite previous efforts to simulate optical tactile sensors, there remain…
Generating Future Observations to Estimate Grasp Success in Cluttered Environments
Daniel Fernandes Gomes, Wenxuan Mou, Paolo Paoletti +1
End-to-end self-supervised models have been proposed for estimating the success of future candidate grasps and video predictive models for generating future observations. However,…
Robotic Perception of Transparent Objects: A Review
Jiaqi Jiang, Guanqun Cao, Jiankang Deng +2
Transparent object perception is a rapidly developing research problem in artificial intelligence. The ability to perceive transparent objects enables robots to achieve higher leve…
CEDex: Cross-Embodiment Dexterous Grasp Generation at Scale from Human-like Contact Representations
Zhiyuan Wu, Rolandos Alexandros Potamias, Xuyang Zhang +3
Cross-embodiment dexterous grasp synthesis refers to adaptively generating and optimizing grasps for various robotic hands with different morphologies. This capability is crucial f…
Learning Stable Robot Grasping with Transformer-based Tactile Control Policies
En Yen Puang, Zechen Li, Chee Meng Chew +2
Measuring grasp stability is an important skill for dexterous robot manipulation tasks, which can be inferred from haptic information with a tactile sensor. Control policies have t…
MagicSkin: Balancing Marker and Markerless Modes in Vision-Based Tactile Sensors with a Translucent Skin
Oluwatimilehin Tijani, Zhuo Chen, Jiankang Deng +1
Vision-based tactile sensors (VBTS) face a fundamental trade-off in marker and markerless design on the tactile skin: opaque ink markers enable measurement of force and tangential…
Sequential Lasso for feature selection with ultra-high dimensional feature space
Shan Luo, Zehua Chen
We propose a novel approach, Sequential Lasso, for feature selection in linear regression models with ultra-high dimensional feature spaces. We investigate in this article the asym…
ViTacGen: Robotic Pushing with Vision-to-Touch Generation
Zhiyuan Wu, Yijiong Lin, Yongqiang Zhao +4
Robotic pushing is a fundamental manipulation task that requires tactile feedback to capture subtle contact forces and dynamics between the end-effector and the object. However, re…
MultiClear: Multimodal Soft Exoskeleton Glove for Transparent Object Grasping Assistance
Chen Hu, Timothy Neate, Shan Luo +1
Grasping is a fundamental skill for interacting with the environment. However, this ability can be difficult for some (e.g. due to disability). Wearable robotic solutions can enhan…