6 papers
τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision
Ning Cheng, Jinan Xu, Wanlin Li +5
Incorporating tactile sensing into Vision-Language-Action (VLA) models holds promise for contact-rich manipulation, where visual observations alone often fail to capture critical c…
SToLa: Self-Adaptive Touch-Language Framework with Tactile Commonsense Reasoning in Open-Ended Scenarios
Ning Cheng, Jinan Xu, Jialing Chen +2
This paper explores the challenges of integrating tactile sensing into intelligent systems for multimodal reasoning, particularly in enabling commonsense reasoning about the open-e…
Towards Comprehensive Multimodal Perception: Introducing the Touch-Language-Vision Dataset
Ning Cheng, You Li, Jing Gao +3
Tactility provides crucial support and enhancement for the perception and interaction capabilities of both humans and robots. Nevertheless, the multimodal research related to touch…
Touch100k: A Large-Scale Touch-Language-Vision Dataset for Touch-Centric Multimodal Representation
Ning Cheng, Changhao Guan, Jing Gao +7
Touch holds a pivotal position in enhancing the perceptual and interactive capabilities of both humans and robots. Despite its significance, current tactile research mainly focuses…
Transformer in Touch: A Survey
Jing Gao, Ning Cheng, Bin Fang +1
The Transformer model, initially achieving significant success in the field of natural language processing, has recently shown great potential in the application of tactile percept…
Potential and Limitations of LLMs in Capturing Structured Semantics: A Case Study on SRL
Ning Cheng, Zhaohui Yan, Ziming Wang +6
Large Language Models (LLMs) play a crucial role in capturing structured semantics to enhance language understanding, improve interpretability, and reduce bias. Nevertheless, an on…