9 citations · 13 across the 15 of their papers we have counts for
10 papers · 1 filter
PhaseWin Search Framework Enable Efficient Object-Level Interpretation
Zihan Gu, Ruoyu Chen, Junchi Zhang +3
Attribution is essential for interpreting object-level foundation models. Recent methods based on submodular subset selection have achieved high faithfulness, but their efficiency…
Did Models Learn Sufficiently? Attribution-Guided Training via Subset-Selected Counterfactual Augmentation
Yannan Chen, Ruoyu Chen, Wei Wang +6
Current visual models often make predictions based on a limited set of discriminative visual cues. As a result, they may become unreliable when the distribution shifts or when thes…
Where MLLMs Attend and What They Rely On: Explaining Autoregressive Token Generation
Ruoyu Chen, Xiaoqing Guo, Kangwei Liu +6
Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in aligning visual inputs with natural language outputs. Yet, the extent to which generated token…
Phantom-Insight: Adaptive Multi-cue Fusion for Video Camouflaged Object Detection with Multimodal LLM
Hua Zhang, Changjiang Luo
Video camouflaged object detection (VCOD) is challenging due to dynamic environments. Existing methods face two main issues: (1) SAM-based methods struggle to separate camouflaged…
SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controlling
Shixuan Sun, Siyuan Liang, Jianjie Huang +2
Retrieval-Augmented Generation (RAG) and its Multimodal Retrieval-Augmented Generation (MRAG) significantly improve the knowledge coverage and contextual understanding of Large Lan…
Benchmarking the Trustworthiness in Multimodal LLMs for Video Understanding
Youze Wang, Zijun Chen, Ruoyu Chen +8
Recent advancements in multimodal large language models for video understanding (videoLLMs) have enhanced their capacity to process complex spatiotemporal data. However, challenges…
Deconstructing Positional Information: From Attention Logits to Training Biases
Zihan Gu, Ruoyu Chen, Han Zhang +2
Positional encodings enable Transformers to incorporate sequential information, yet their theoretical understanding remains limited to two properties: distance attenuation and tran…
FaceInsight: A Multimodal Large Language Model for Face Perception
Jingzhi Li, Changjiang Luo, Ruoyu Chen +4
Recent advances in multimodal large language models (MLLMs) have demonstrated strong capabilities in understanding general visual content. However, these general-domain MLLMs perfo…
Beyond Progress Measures: Theoretical Insights into the Mechanism of Grokking
Zihan Gu, Ruoyu Chen, Hua Zhang +2
Grokking, referring to the abrupt improvement in test accuracy after extended overfitting, offers valuable insights into the mechanisms of model generalization. Existing researches…
Generalized Semantic Contrastive Learning via Embedding Side Information for Few-Shot Object Detection
Ruoyu Chen, Hua Zhang, Jingzhi Li +3
The objective of few-shot object detection (FSOD) is to detect novel objects with few training samples. The core challenge of this task is how to construct a generalized feature sp…