13 papers
Rethinking Gating Mechanism in Sparse MoE: Handling Arbitrary Modality Inputs with Confidence-Guided Gate
Liangwei Nathan Zheng, Wei Emma Zhang, Mingyu Guo +2
Effectively managing missing modalities is a fundamental challenge in real-world multimodal learning scenarios, where data incompleteness often results from systematic collection e…
Test-Time Attention Purification for Backdoored Large Vision Language Models
Zhifang Zhang, Bojun Yang, Shuo He +5
Despite the strong multimodal performance, large vision-language models (LVLMs) are vulnerable during fine-tuning to backdoor attacks, where adversaries insert trigger-embedded sam…
Calibration Attention: Learning Reliability-Aware Representations for Vision Transformers
Wenhao Liang, Wei Emma Zhang, Lin Yue +4
Most calibration methods operate at the logit level, implicitly assuming that miscalibration can be corrected without changing the underlying representation. We challenge this assu…
Lifting Manifolds to Mitigate Pseudo-Alignment in LLM4TS
Liangwei Nathan Zheng, Wenhao Liang, Wei Emma Zhang +3
Pseudo-Alignment is a pervasive challenge in many large language models for time series (LLM4TS) models, often causing them to underperform compared to linear models or randomly in…
MMiC: Mitigating Modality Incompleteness in Clustered Federated Learning
Lishan Yang, Wei Emma Zhang, Quan Z. Sheng +3
In the era of big data, data mining has become indispensable for uncovering hidden patterns and insights from vast and complex datasets. The integration of multimodal data sources…
Calibrating Deep Neural Network using Euclidean Distance
Wenhao Liang, Chang Dong, Liangwei Zheng +2
Uncertainty is a fundamental aspect of real-world scenarios, where perfect information is rarely available. Humans naturally develop complex internal models to navigate incomplete…