36 citations · 49 across the 18 of their papers we have counts for
21 papers
PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents
Siqi Yang, Yimu Wang, Ziqi Cai +6
A multiplayer world model must predict how the independent actions of several players change a shared environment, including changes outside a player's current field of view. A poi…
LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering
Xuye Liu, Yimu Wang, Peng Shi +7
Scientific literature is increasingly used as a knowledge source for language models, retrieval-augmented generation systems, and research assistants, but answering research questi…
Where Does the Answer Come From? Benchmarking View-Level Visual Evidence Identification in Multi-View MLLMs for Autonomous Driving
Yimu Wang, Yee Man Choi, Barry Zhang +3
Multimodal large language models (MLLMs) achieve strong results on visual reasoning benchmarks, but answer accuracy alone does not indicate whether a model relied on the correct vi…
VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence
Mozhgan Nasr Azadani, Yimu Wang, Yongpeng Zhu +5
Establishing a clear link between model predictions and the visual evidence that supports them is critical for transparency and reliability in multimodal reasoning, yet current mul…
UNIFORM: Unifying Knowledge from Large-scale and Diverse Pre-trained Models
Yimu Wang, Weiming Zhuang, Chen Chen +3
In the era of deep learning, the increasing number of pre-trained models available online presents a wealth of knowledge. These models, developed with diverse architectures and tra…
HAWAII: Hierarchical Visual Knowledge Transfer for Efficient Vision-Language Models
Yimu Wang, Mozhgan Nasr Azadani, Sean Sedwards +1
Improving the visual understanding ability of vision-language models (VLMs) is crucial for enhancing their performance across various tasks. While using multiple pretrained visual…
Survey of Video Diffusion Models: Foundations, Implementations, and Applications
Yimu Wang, Xuye Liu, Wei Pang +4
Recent advances in diffusion models have revolutionized video generation, offering superior temporal consistency and visual quality compared to traditional generative adversarial n…
LEO-MINI: An Efficient Multimodal Large Language Model using Conditional Token Reduction and Mixture of Multi-Modal Experts
Yimu Wang, Mozhgan Nasr Azadani, Sean Sedwards +1
Redundancy of visual tokens in multi-modal large language models (MLLMs) significantly reduces their computational efficiency. Recent approaches, such as resamplers and summarizers…
OV-SCAN: Semantically Consistent Alignment for Novel Object Discovery in Open-Vocabulary 3D Object Detection
Adrian Chow, Evelien Riddell, Yimu Wang +2
Open-vocabulary 3D object detection for autonomous driving aims to detect novel objects beyond the predefined training label sets in point cloud scenes. Existing approaches achieve…
Mitigating the Modality Gap: Few-Shot Out-of-Distribution Detection with Multi-modal Prototypes and Image Bias Estimation
Yimu Wang, Evelien Riddell, Adrian Chow +2
Existing vision-language model (VLM)-based methods for out-of-distribution (OOD) detection typically rely on similarity scores between input images and in-distribution (ID) text pr…
Rethinking Spectral Augmentation for Contrast-based Graph Self-Supervised Learning
Xiangru Jian, Xinjian Zhao, Wei Pang +4
The recent surge in contrast-based graph self-supervised learning has prominently featured an intensified exploration of spectral cues. Spectral augmentation, which involves modify…
DREAM: Improving Video-Text Retrieval Through Relevance-Based Augmentation Using Large Foundation Models
Yimu Wang, Shuai Yuan, Bo Xue +4
Recent progress in video-text retrieval has been driven largely by advancements in model architectures and training strategies. However, the representation learning capabilities of…
Efficient Algorithms for Generalized Linear Bandits with Heavy-tailed Rewards
Bo Xue, Yimu Wang, Yuanyu Wan +2
This paper investigates the problem of generalized linear bandits with heavy-tailed rewards, whose -th moment is bounded for some . Although there exist methods…
InvGC: Robust Cross-Modal Retrieval by Inverse Graph Convolution
Xiangru Jian, Yimu Wang
Over recent decades, significant advancements in cross-modal retrieval are mainly driven by breakthroughs in visual and linguistic modeling. However, a recent study shows that mult…
Balance Act: Mitigating Hubness in Cross-Modal Retrieval with Query and Gallery Banks
Yimu Wang, Xiangru Jian, Bo Xue
In this work, we present a post-processing solution to address the hubness problem in cross-modal retrieval, a phenomenon where a small number of gallery data points are frequently…
NICE: CVPR 2023 Challenge on Zero-shot Image Captioning
Taehoon Kim, Pyunghwan Ahn, Sangyun Kim +39
In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge. This project is designed t…
Cooperation or Competition: Avoiding Player Domination for Multi-Target Robustness via Adaptive Budgets
Yimu Wang, Dinghuai Zhang, Yihan Wu +2
Despite incredible advances, deep learning has been shown to be susceptible to adversarial attacks. Numerous approaches have been proposed to train robust networks both empirically…
Video-Text Retrieval by Supervised Sparse Multi-Grained Learning
Yimu Wang, Peng Shi
While recent progress in video-text retrieval has been advanced by the exploration of better representation learning, in this paper, we present a novel multi-grained sparse learnin…
Multimodal Federated Learning via Contrastive Representation Ensemble
Qiying Yu, Yang Liu, Yimu Wang +2
With the increasing amount of multimedia data on modern mobile systems and IoT infrastructures, harnessing these rich multimodal data without breaching user privacy becomes a criti…
Multi-View Fusion Transformer for Sensor-Based Human Activity Recognition
Yimu Wang, Kun Yu, Yan Wang +1
As a fundamental problem in ubiquitous computing and machine learning, sensor-based human activity recognition (HAR) has drawn extensive attention and made great progress in recent…
Nearly Optimal Regret for Stochastic Linear Bandits with Heavy-Tailed Payoffs
Bo Xue, Guanghui Wang, Yimu Wang +1
In this paper, we study the problem of stochastic linear bandits with finite action sets. Most of existing work assume the payoffs are bounded or sub-Gaussian, which may be violate…