9 papers
CertBind from Multimodal Connectivity to Certifiable Retrieval Decisions
Shuheng Cao, Zhenhao Zhang, Ruiqi Chen +7
Lightweight connectors make frozen multimodal encoders composable at the representation level. Deployment exposes a second problem at the level of task decisions. A connected route…
SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents
Yang Wan, Zhenhao Zhang, Jierui Wang +2
Deciding whether a trajectory actually fulfills its instruction determines how we measure computer-use agents on long-horizon graphical-user-interface tasks and how we train them w…
Event-VLA: Action-Conditioned Event Fusion for Robust Vision-Language-Action Model
Jiaxin Liu, Xun Xu, Zhenhao Zhang +5
Vision-Language-Action (VLA) models have become an important paradigm of embodied AI. However, existing VLA models typically assume well-lit and stable indoor settings, while real-…
Mitigating Conversational Inertia in Multi-Turn Agents
Yang Wan, Zheng Cao, Zhenhao Zhang +4
Large language models excel as few-shot learners when provided with appropriate demonstrations, yet this strength becomes problematic in multiturn agent scenarios, where LLMs erron…
Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards
Ming Li, Pei Chen, Zhenhao Zhang +10
Large Language Models demonstrate strong capabilities in single-turn instruction following but suffer from Lost-in-Conversation (LiC), a degradation in performance as information i…
HOID-R1: Reinforcement Learning for Open-World Human-Object Interaction Detection Reasoning with Multimodal Large Language Model
Zhenhao Zhang, Hanqing Wang, Xiangyu Zeng +10
Understanding and recognizing human-object interaction (HOI) is a pivotal application in AR/VR and robotics. Recent open-vocabulary HOI detection approaches depend exclusively on l…
Generalizable Operating Room Expert with Multimodal Enhancement
Peiqi He, Zhenhao Zhang, Yixiang Zhang +3
Precise spatial modeling in the operating room (OR) is essential for intraoperative awareness, hazard avoidance, and surgical decision-making. Although existing approaches exploit…
SDEval: Safety Dynamic Evaluation for Multimodal Large Language Models
Hanqing Wang, Yuan Tian, Mingyu Liu +2
In the rapidly evolving landscape of Multimodal Large Language Models (MLLMs), the safety concerns of their outputs have earned significant attention. Although numerous datasets ha…
Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learning
Hanqing Wang, Zhenhao Zhang, Kaiyang Ji +12
3D affordance grounding aims to understand how diverse objects can be manipulated, making it a cornerstone of embodied interaction. However, prior works struggle to generalize to o…