10 papers
Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals
Jiyuan Liu, Liangwei Nathan Zheng, Wei Emma Zhang +2
Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed. A modality that is useful for one input…
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
Liangwei Nathan Zheng, Wei Emma Zhang, Olaf Maennel +2
Mixture-of-Experts (MoE) presents a naturally compatible and scalable framework for multimodal learning, demonstrating strong adaptability across diverse modalities and tasks. Desp…
Rethinking Gating Mechanism in Sparse MoE: Handling Arbitrary Modality Inputs with Confidence-Guided Gate
Liangwei Nathan Zheng, Wei Emma Zhang, Mingyu Guo +2
Effectively managing missing modalities is a fundamental challenge in real-world multimodal learning scenarios, where data incompleteness often results from systematic collection e…
Lifting Manifolds to Mitigate Pseudo-Alignment in LLM4TS
Liangwei Nathan Zheng, Wenhao Liang, Wei Emma Zhang +3
Pseudo-Alignment is a pervasive challenge in many large language models for time series (LLM4TS) models, often causing them to underperform compared to linear models or randomly in…
Focal Calibration Loss: Controlling Posterior Distortion in Deep Neural Classifiers
Wenhao Liang, Chang Dong, Liangwei Zheng +2
Confidence calibration matters wherever a classifier's probabilities, not just its labels, are consumed downstream. We study Focal Calibration Loss (FCL), which adds a squared prob…
Understanding Why Large Language Models Can Be Ineffective in Time Series Analysis: The Impact of Modality Alignment
Liangwei Nathan Zheng, Chang George Dong, Wei Emma Zhang +4
Large Language Models (LLMs) have demonstrated impressive performance in time series analysis and seems to understand the time temporal relationship well than traditional transform…