6 papers
EgoExplorer: A Vision-Motion-Touch Observatory for Human Loco-Manipulation In Situ
Yunsong Zhou, Xuekun Jiang, Haiqi Shen +11
Learning humanoid loco-manipulation from human demonstrations rests on observing how locomotion, posture, and object interaction evolve together as tasks unfold. Scaling collection…
Gestalt: Large Multimodal Interplay Model
Zequn Yang, Yu Miao, Haotian Ni +8
In this paper, we propose Gestalt, a new paradigm of large multimodal model built around multimodal interplay. Despite rapid advances, large multimodal models are reaching a bottle…
AnyTouch 2: General Optical Tactile Representation Learning For Dynamic Tactile Perception
Ruoxuan Feng, Yuxuan Zhou, Siyu Mei +6
Real-world contact-rich manipulation demands robots to perceive temporal tactile feedback, capture subtle surface deformations, and reason about object properties as well as force…
Object-AVEdit: An Object-level Audio-Visual Editing Model
Youquan Fu, Ruiyang Si, Hongfa Wang +6
There is a high demand for audio-visual editing in video post-production and the film making field. While numerous models have explored audio and video editing, they struggle with…
MokA: Multimodal Low-Rank Adaptation for MLLMs
Yake Wei, Yu Miao, Dongzhan Zhou +1
In this paper, we reveal that most current efficient multimodal fine-tuning methods are hindered by a key limitation: they are directly borrowed from LLMs, often neglecting the int…
Ref-AVS: Refer and Segment Objects in Audio-Visual Scenes
Yaoting Wang, Peiwen Sun, Dongzhan Zhou +3
Traditional reference segmentation tasks have predominantly focused on silent visual scenes, neglecting the integral role of multimodal perception and interaction in human experien…