9 papers
AeroGround: A Comprehensive Benchmark for Aerial-Ground Collaborative Reasoning
Shenghong Yi, Lin Zhang, Muzian Li +6
Vision-language models (VLMs) have been widely employed in understanding and reasoning tasks for unmanned aerial vehicles (UAVs). Existing UAV benchmarks primarily focus on aerial-…
Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System
Haoyu Zhang, Shuoxun Zhang, Peng Ye +5
Multimodal Large Language Model (MLLM)-based UAV aerial image understanding and reasoning is essential for aerial intelligence yet poses distinct challenges arising from extreme sc…
MLG-Stereo: ViT Based Stereo Matching with Multi-Stage Local-Global Enhancement
Haoyu Zhang, Jingyi Zhou, Peng Ye +4
With the development of deep learning, ViT-based stereo matching methods have made significant progress due to their remarkable robustness and zero-shot ability. However, due to th…
FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understanding
Kangcong Li, Peng Ye, Lin Zhang +4
Transitioning Multimodal Large Language Models (MLLMs) from offline to online streaming video understanding is essential for continuous perception. However, existing methods lack f…
SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning
Lin Zhang, Xianfang Zeng, Kangcong Li +2
We propose SC-Captioner, a reinforcement learning framework that enables the self-correcting capability of image caption models. Our crucial technique lies in the design of the rew…
PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding
Kangcong Li, Peng Ye, Chongjun Tu +6
While Large Language Models (LLMs) demonstrate strong performance across domains, their long-context capabilities are limited by transient neural activations causing information de…
FAVOR-Bench: A Comprehensive Benchmark for Fine-Grained Video Motion Understanding
Chongjun Tu, Lin Zhang, Pengtao Chen +5
Multimodal Large Language Models (MLLMs) have shown remarkable capabilities in video content understanding but still struggle with fine-grained motion comprehension. To comprehensi…
TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models
Xudong Tan, Peng Ye, Chongjun Tu +5
Multimodal Large Language Models (MLLMs) are becoming increasingly popular, while the high computational cost associated with multimodal data input, particularly from visual tokens…
DeRS: Towards Extremely Efficient Upcycled Mixture-of-Experts Models
Yongqi Huang, Peng Ye, Chenyu Huang +5
Upcycled Mixture-of-Experts (MoE) models have shown great potential in various tasks by converting the original Feed-Forward Network (FFN) layers in pre-trained dense models into M…