5 papers
TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning
Muyu Pan, Shu Zhao, Nan Zhang +4
This paper investigates large language model (LLM) abstention learning, specifically using ternary reward, which incentivize truthfulness in large language models. This paper exten…
When Relations Break: Analyzing Relation Hallucination in Vision-Language Model Under Rotation and Noise
Philip Wootaek Shin, Ajay Narayanan Sridhar, Sivani Devarapalli +3
Vision-language models (VLMs) achieve strong multimodal performance but remain prone to relation hallucination, which requires accurate reasoning over inter-object interactions. We…
Parts-Mamba: Augmenting Joint Context with Part-Level Scanning for Occluded Human Skeleton
Tianyi Shen, Huijuan Xu, Nilesh Ahuja +3
Skeleton action recognition involves recognizing human action from human skeletons. The use of graph convolutional networks (GCNs) has driven major advances in this recognition tas…
MoRA: Missing Modality Low-Rank Adaptation for Visual Recognition
Shu Zhao, Nilesh Ahuja, Tan Yu +2
Pre-trained vision language models have shown remarkable performance on visual recognition tasks, but they typically assume the availability of complete multimodal inputs during bo…
Windsock is Dancing: Adaptive Multimodal Retrieval-Augmented Generation
Shu Zhao, Tianyi Shen, Nilesh Ahuja +2
Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a promising method to generate factual and up-to-date responses of Multimodal Large Language Models (MLLMs) by incor…