24 citations · 24 across the 18 of their papers we have counts for
20 papers
Region-Level Policy Optimization for Fine-grained MLLM Perception
Yuheng Shi, Xiaohuan Pei, Minjing Dong +1
Fine-grained visual perception in MLLMs is commonly improved by raising the resolution, but the added visual tokens inflate vision-encoding and language-model prefilling costs. We…
Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models
Yuheng Shi, Xiaohuan Pei, Linfeng Wen +2
MLLMs require high-resolution visual inputs for fine-grained tasks like document understanding and dense scene perception. However, current global resolution scaling paradigms indi…
Confidence Calibration under Ambiguous Ground Truth
Linwei Tao, Haoyang Luo, Minjing Dong +1
Confidence calibration assumes a unique ground-truth label per input, yet this assumption fails wherever annotators genuinely disagree. Post-hoc calibrators fitted on majority-vote…
PA-Attack: Guiding Gray-Box Attacks on LVLM Vision Encoders with Prototypes and Attention
Hefei Mei, Zirui Wang, Chang Xu +2
Large Vision-Language Models (LVLMs) are foundational to modern multimodal applications, yet their susceptibility to adversarial attacks remains a critical concern. Prior white-box…
Can Large Language Models Express Uncertainty Like Human?
Linwei Tao, Yi-Fan Yeh, Bo Kai +6
Large language models (LLMs) are increasingly used in high-stakes settings, where overconfident responses can mislead users. Reliable confidence estimation has been shown to enhanc…
Catching the Details: Self-Distilled RoI Predictors for Fine-Grained MLLM Perception
Yuheng Shi, Xiaohuan Pei, Minjing Dong +1
Multimodal Large Language Models (MLLMs) require high-resolution visual information to perform fine-grained perception, yet processing entire high-resolution images is computationa…
WATS: Calibrating Graph Neural Networks with Wavelet-Aware Temperature Scaling
Xiaoyang Li, Linwei Tao, Haohui Lu +3
Graph Neural Networks (GNNs) have demonstrated strong predictive performance on relational data; however, their confidence estimates often misalign with actual predictive correctne…
Sample Margin-Aware Recalibration of Temperature Scaling
Haolan Guo, Linwei Tao, Haoyang Luo +2
Recent advances in deep learning have significantly improved predictive accuracy. However, modern neural networks remain systematically overconfident, posing risks for deployment i…
Revisiting Uncertainty Estimation and Calibration of Large Language Models
Linwei Tao, Yi-Fan Yeh, Minjing Dong +3
As large language models (LLMs) are increasingly deployed in high-stakes applications, robust uncertainty estimation is essential for ensuring the safe and trustworthy deployment o…
Beyond One-Hot Labels: Semantic Mixing for Model Calibration
Haoyang Luo, Linwei Tao, Minjing Dong +1
Model calibration seeks to ensure that models produce confidence scores that accurately reflect the true likelihood of their predictions being correct. However, existing calibratio…
VEAttack: Downstream-agnostic Vision Encoder Attack against Large Vision Language Models
Hefei Mei, Zirui Wang, Shen You +2
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding and generation, yet their vulnerability to adversarial attacks raises sig…
Uncertainty Weighted Gradients for Model Calibration
Jinxu Lin, Linwei Tao, Minjing Dong +1
Model calibration is essential for ensuring that the predictions of deep neural networks accurately reflect true probabilities in real-world classification tasks. However, deep net…
Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibration
Younan Zhu, Linwei Tao, Minjing Dong +1
Large Vision-Language Models (LVLMs) exhibit impressive multimodal reasoning capabilities but remain highly susceptible to object hallucination, where models generate responses tha…
Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentation
Yuheng Shi, Minjing Dong, Chang Xu
While Contrastive Language-Image Pre-training (CLIP) has advanced open-vocabulary predictions, its performance on semantic segmentation remains suboptimal. This shortfall primarily…
Feature Clipping for Uncertainty Calibration
Linwei Tao, Minjing Dong, Chang Xu
Deep neural networks (DNNs) have achieved significant success across various tasks, but ensuring reliable uncertainty estimates, known as model calibration, is crucial for their sa…
Consistency Calibration: Improving Uncertainty Calibration via Consistency among Perturbed Neighbors
Linwei Tao, Haolan Guo, Minjing Dong +1
Calibration is crucial in deep learning applications, especially in fields like healthcare and autonomous driving, where accurate confidence estimates are vital for decision-making…
Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models
Jinxu Lin, Linwei Tao, Minjing Dong +1
As diffusion models become increasingly popular, the misuse of copyrighted and private images has emerged as a major concern. One promising solution to mitigate this issue is ident…
VSSD: Vision Mamba with Non-Causal State Space Duality
Yuheng Shi, Minjing Dong, Mingjia Li +1
Vision transformers have significantly advanced the field of computer vision, offering robust modeling capabilities and global receptive field. However, their high computational de…
Efficient Image-to-Image Diffusion Classifier for Adversarial Robustness
Hefei Mei, Minjing Dong, Chang Xu
Diffusion models (DMs) have demonstrated great potential in the field of adversarial robustness, where DM-based defense methods can achieve superior defense capability without adve…
Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model
Yuheng Shi, Minjing Dong, Chang Xu
Despite the significant achievements of Vision Transformers (ViTs) in various vision tasks, they are constrained by the quadratic complexity. Recently, State Space Models (SSMs) ha…