7 citations · 20 across the 14 of their papers we have counts for
15 papers
SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models
Shiyu Li, Zi-Yuan Hu, Shijia Huang +3
Despite their strong multimodal understanding ability, multimodal large language models (MLLMs) incur substantial computational overhead when processing long visual token sequences…
VideoLatent: Video-Language Learning via Latent Self-Forcing
Zi-Yuan Hu, Zicong Tang, Shijia Huang +3
Recent advancements in chain-of-thought (CoT) reasoning have shown promise in enhancing video understanding and reasoning capabilities of multimodal large language models (MLLMs).…
Efficient-VLN: A Simple yet Strong Baseline for Efficient Vision-Language Navigation
Duo Zheng, Shijia Huang, Yanyang Li +1
While Multimodal Large Language Models (MLLMs) have demonstrated significant promise in Vision-Language Navigation (VLN), existing agents remain heavily constrained by systemic bot…
NeMo: Needle in a Montage for Video-Language Understanding
Zi-Yuan Hu, Shuo Liang, Duo Zheng +10
Recent advances in video large language models (VideoLLMs) call for new evaluation protocols and benchmarks for video-language understanding. Inspired by the needle in a haystack t…
Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors
Duo Zheng, Shijia Huang, Yanyang Li +1
Previous research has investigated the application of Multimodal Large Language Models (MLLMs) in understanding 3D scenes by interpreting them as videos. These approaches generally…
Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding
Duo Zheng, Shijia Huang, Liwei Wang
The rapid advancement of Multimodal Large Language Models (MLLMs) has significantly impacted various multimodal tasks. However, these models face challenges in tasks that require s…
Enhancing Temporal Modeling of Video LLMs via Time Gating
Zi-Yuan Hu, Yiwu Zhong, Shijia Huang +2
Video Large Language Models (Video LLMs) have achieved impressive performance on video-and-language tasks, such as video question answering. However, most existing Video LLMs negle…
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
Hao Zhang, Hongyang Li, Feng Li +8
With the recent significant advancements in large multi-modal models (LMMs), the importance of their grounding capability in visual chat is increasingly recognized. Despite recent…
Towards Learning a Generalist Model for Embodied Navigation
Duo Zheng, Shijia Huang, Lin Zhao +2
Building a generalist agent that can interact with the world is the intriguing target of AI systems, thus spurring the research for embodied navigation, where an agent is required…
CLEVA: Chinese Language Models EVAluation Platform
Yanyang Li, Jianqiao Zhao, Duo Zheng +8
With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model's capabilities has become an increasingly significant issue. The absence of a compreh…
MP-Former: Mask-Piloted Transformer for Image Segmentation
Hao Zhang, Feng Li, Huaizhe Xu +4
We present a mask-piloted Transformer which improves masked-attention in Mask2Former for image segmentation. The improvement is based on our observation that Mask2Former suffers fr…
DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding
Shilong Liu, Yaoyuan Liang, Feng Li +5
In this paper, we study the problem of visual grounding by considering both phrase extraction and grounding (PEG). In contrast to the previous phrase-known-at-test setting, PEG req…
A Unified Mutual Supervision Framework for Referring Expression Segmentation and Generation
Shijia Huang, Feng Li, Hao Zhang +3
Reference Expression Segmentation (RES) and Reference Expression Generation (REG) are mutually inverse tasks that can be naturally jointly trained. Though recent work has explored…
Multi-View Transformer for 3D Visual Grounding
Shijia Huang, Yilun Chen, Jiaya Jia +1
The 3D visual grounding task aims to ground a natural language description to the targeted object in a 3D scene, which is usually represented in 3D point clouds. Previous works stu…
DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectors
Yilun Chen, Shijia Huang, Shu Liu +2
Camera-based 3D object detectors are welcome due to their wider deployment and lower price than LiDAR sensors. We first revisit the prior stereo detector DSGN for its stereo volume…