86 citations · 86 across the 12 of their papers we have counts for
15 papers
Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluation
Shangzhe Di, Zhaokai Wang, Weidi Xie
Recent work, such as Vision Banana, shows that lightweight instruction tuning can enable an image generator to achieve state-of-the-art performance across multiple visual perceptio…
WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting
Zhaokai Wang, Tianlin Gui, Jiayuan Rao +3
Predicting a football match before kickoff requires more than knowing past results: a model must use changing information and make a clear prediction before the answer is available…
Towards Comprehensive Basketball Understanding
Yirong Hu, Jiayuan Rao, Yu Zhang +2
Understanding a basketball game requires recognizing events, localizing actions, identifying players, and relating these to structured game knowledge. Existing benchmarks primarily…
OmniStream: Mastering Perception, Reconstruction and Action in Continuous Streams
Yibin Yan, Jilan Xu, Shangzhe Di +2
Modern visual agents require representations that are general, causal, and physically structured to operate in real-time streaming environments. However, current vision foundation…
VersaViT: Enhancing MLLM Vision Backbones via Task-Guided Optimization
Yikun Liu, Yuan Liu, Shangzhe Di +8
Multimodal Large Language Models (MLLMs) have recently achieved remarkable success in visual-language understanding, demonstrating superior high-level semantic alignment within the…
Weaver: End-to-End Agentic System Training for Video Interleaved Reasoning
Yudi Shi, Shangzhe Di, Qirui Chen +5
Video reasoning constitutes a comprehensive assessment of a model's capabilities, as it demands robust perceptual and interpretive skills, thereby serving as a means to explore the…
MTV: Revisiting Multi-Task Visual Representation Learning
Shangzhe Di, Zhonghua Zhai, Weidi Xie
Current visual representation learning remains bifurcated: vision-language models (e.g., CLIP) excel at global semantic alignment but lack spatial precision, while self-supervised…
Universal Video Temporal Grounding with Generative Multi-modal Large Language Models
Zeqian Li, Shangzhe Di, Zhonghua Zhai +3
This paper presents a computational model for universal video temporal grounding, which accurately localizes temporal moments in videos based on natural language queries (e.g., que…
Learning Streaming Video Representation via Multitask Training
Yibin Yan, Jilan Xu, Shangzhe Di +6
Understanding continuous video streams plays a fundamental role in real-time applications including embodied AI and autonomous driving. Unlike offline video understanding, streamin…
Streaming Video Question-Answering with In-context Video KV-Cache Retrieval
Shangzhe Di, Zhelun Yu, Guanghao Zhang +7
We propose ReKV, a novel training-free approach that enables efficient streaming video question-answering (StreamingVQA), by seamlessly integrating with existing Video Large Langua…
Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillation
Yudi Shi, Shangzhe Di, Qirui Chen +1
This paper tackles the problem of video question answering (VideoQA), a task that often requires multi-step reasoning and a profound understanding of spatial-temporal dynamics. Whi…
Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos
Qirui Chen, Shangzhe Di, Weidi Xie
This paper considers the problem of Multi-Hop Video Question Answering (MH-VidQA) in long-form egocentric videos. This task not only requires to answer visual questions, but also t…
Grounded Question-Answering in Long Egocentric Videos
Shangzhe Di, Weidi Xie
Existing approaches to video understanding, mainly designed for short videos from a third-person perspective, are limited in their applicability in certain fields, such as robotics…
Sparse Dense Fusion for 3D Object Detection
Yulu Gao, Chonghao Sima, Shaoshuai Shi +3
With the prevalence of multimodal learning, camera-LiDAR fusion has gained popularity in 3D object detection. Although multiple fusion approaches have been proposed, they can be cl…
Video Background Music Generation with Controllable Music Transformer
Shangzhe Di, Zeren Jiang, Si Liu +5
In this work, we address the task of video background music generation. Some previous works achieve effective music generation but are unable to generate melodious music tailored t…