1 citations · 1 across the 10 of their papers we have counts for
12 papers
Rethinking Multi-Image Re-Representation in Multi-Image Understanding
Gengyuan Zhang, Xiao Han, Xinyu Xie +2
Multi-image understanding requires MLLMs not only to recognise the content of individual images, but also to organise visual evidence distributed across them. We study this problem…
Conformalized Large Language Models under Configuration Shift
Yuqicheng Zhu, Jialin Yu, Lin Li +7
Conformal prediction (CP) is a distribution-free framework for uncertainty quantification that has recently been adapted to large language models (LLMs), providing prediction sets…
ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replay
Gengyuan Zhang, Mingcong Ding, Jingpei Wu +2
Embodied exploration is a target-driven process that requires embodied agents to possess fine-grained perception and knowledge-enhanced decision making. While recent attempts lever…
AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interaction
Gengyuan Zhang, Tanveer Hannan, Hermine Kleiner +6
An ideal vision-language agent serves as a bridge between the human users and their surrounding physical world in real-world applications like autonomous driving and embodied agent…
Unveiling the "Fairness Seesaw": Discovering and Mitigating Gender and Race Bias in Vision-Language Models
Jian Lan, Udo Schlegel, Tanveer Hannan +3
Although Vision-Language Models (VLMs) have achieved remarkable success, the knowledge mechanisms underlying their social biases remain a black box, where fairness- and ethics-rela…
Memory Helps, but Confabulation Misleads: Understanding Streaming Events in Videos with MLLMs
Gengyuan Zhang, Mingcong Ding, Tong Liu +2
Multimodal large language models (MLLMs) have demonstrated strong performance in understanding videos holistically, yet their ability to process streaming videos-videos are treated…
Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
Roberto Amoroso, Gengyuan Zhang, Rajat Koner +3
Video Question Answering (Video QA) is a challenging video understanding task that requires models to comprehend entire videos, identify the most relevant information based on cont…
VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs
Ruotong Liao, Max Erler, Huiyu Wang +4
In the video-language domain, recent works in leveraging zero-shot Large Language Model-based reasoning for video understanding have become competitive challengers to previous end-…
FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models
Haokun Chen, Hang Li, Yao Zhang +7
One-Shot Federated Learning (OSFL), a special decentralized machine learning paradigm, has recently gained significant attention. OSFL requires only a single round of client data o…
Multimodal Pragmatic Jailbreak on Text-to-image Models
Tong Liu, Zhixin Lai, Jiawen Wang +6
Diffusion models have recently achieved remarkable advancements in terms of image quality and fidelity to textual prompts. Concurrently, the safety of such generative models has be…
Localizing Events in Videos with Multimodal Queries
Gengyuan Zhang, Mang Ling Ada Fok, Jialu Ma +5
Localizing events in videos based on semantic queries is a pivotal task in video understanding, with the growing significance of user-oriented applications like video search. Yet,…
SPOT! Revisiting Video-Language Models for Event Understanding
Gengyuan Zhang, Jinhe Bi, Jindong Gu +2
Understanding videos is an important research topic for multimodal learning. Leveraging large-scale datasets of web-crawled video-text pairs as weak supervision has become a pre-tr…