2 citations · 6 across the 13 of their papers we have counts for
15 papers
WorldBridge: Bridging Video and Action for Task Generalization in World Action Models
Jun Cen, Yubo Guo, Hao Li +9
Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (W…
Data Pyramid for Embodied Manipulation: A Survey
Yifan Ye, Yankai Fu, Yaoxu Lv +26
Multimodal foundation models learned to see and to speak by consuming the whole internet. Embodied agents admit no such shortcut, since they require data that couple observations w…
Efficient-VLN: A Simple yet Strong Baseline for Efficient Vision-Language Navigation
Duo Zheng, Shijia Huang, Yanyang Li +1
While Multimodal Large Language Models (MLLMs) have demonstrated significant promise in Vision-Language Navigation (VLN), existing agents remain heavily constrained by systemic bot…
Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors
Duo Zheng, Shijia Huang, Yanyang Li +1
Previous research has investigated the application of Multimodal Large Language Models (MLLMs) in understanding 3D scenes by interpreting them as videos. These approaches generally…
CLEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation
Yanyang Li, Tin Long Wong, Cheung To Hung +5
Recent advances in large language models (LLMs) have shown significant promise, yet their evaluation raises concerns, particularly regarding data contamination due to the lack of a…
Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding
Duo Zheng, Shijia Huang, Liwei Wang
The rapid advancement of Multimodal Large Language Models (MLLMs) has significantly impacted various multimodal tasks. However, these models face challenges in tasks that require s…
Towards Learning a Generalist Model for Embodied Navigation
Duo Zheng, Shijia Huang, Lin Zhao +2
Building a generalist agent that can interact with the world is the intriguing target of AI systems, thus spurring the research for embodied navigation, where an agent is required…
CLEVA: Chinese Language Models EVAluation Platform
Yanyang Li, Jianqiao Zhao, Duo Zheng +8
With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model's capabilities has become an increasingly significant issue. The absence of a compreh…
Towards Unifying Multi-Lingual and Cross-Lingual Summarization
Jiaan Wang, Fandong Meng, Duo Zheng +4
To adapt text summarization to the multilingual world, previous work proposes multi-lingual summarization (MLS) and cross-lingual summarization (CLS). However, these two tasks have…
Towards Unifying Reference Expression Generation and Comprehension
Duo Zheng, Tao Kong, Ya Jing +2
Reference Expression Generation (REG) and Comprehension (REC) are two highly correlated tasks. Modeling REG and REC simultaneously for utilizing the relation between them is a prom…
A Survey on Cross-Lingual Summarization
Jiaan Wang, Fandong Meng, Duo Zheng +4
Cross-lingual summarization is the task of generating a summary in one language (e.g., English) for the given document(s) in a different language (e.g., Chinese). Under the globali…
ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization
Jiaan Wang, Fandong Meng, Ziyao Lu +4
We present ClidSum, a benchmark dataset for building cross-lingual summarization systems on dialogue documents. It consists of 67k+ dialogue documents from two subsets (i.e., SAMSu…
Knowledge Enhanced Sports Game Summarization
Jiaan Wang, Zhixu Li, Tingyi Zhang +5
Sports game summarization aims at generating sports news from live commentaries. However, existing datasets are all constructed through automated collection and cleaning processes,…
Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser
Duo Zheng, Zipeng Xu, Fandong Meng +3
Considering the importance of building a good Visual Dialog (VD) Questioner, many researchers study the topic under a Q-Bot-A-Bot image-guessing game setting, where the Questioner…
Modeling Explicit Concerning States for Reinforcement Learning in Visual Dialogue
Zipeng Xu, Fandong Meng, Xiaojie Wang +3
To encourage AI agents to conduct meaningful Visual Dialogue (VD), the use of Reinforcement Learning has been proven potential. In Reinforcement Learning, it is crucial to represen…