6 papers
Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals
Jiyuan Liu, Liangwei Nathan Zheng, Wei Emma Zhang +2
Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed. A modality that is useful for one input…
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
Liangwei Nathan Zheng, Wei Emma Zhang, Olaf Maennel +2
Mixture-of-Experts (MoE) presents a naturally compatible and scalable framework for multimodal learning, demonstrating strong adaptability across diverse modalities and tasks. Desp…
Rethinking Gating Mechanism in Sparse MoE: Handling Arbitrary Modality Inputs with Confidence-Guided Gate
Liangwei Nathan Zheng, Wei Emma Zhang, Mingyu Guo +2
Effectively managing missing modalities is a fundamental challenge in real-world multimodal learning scenarios, where data incompleteness often results from systematic collection e…
Lifting Manifolds to Mitigate Pseudo-Alignment in LLM4TS
Liangwei Nathan Zheng, Wenhao Liang, Wei Emma Zhang +3
Pseudo-Alignment is a pervasive challenge in many large language models for time series (LLM4TS) models, often causing them to underperform compared to linear models or randomly in…
Calibrating Deep Neural Network using Euclidean Distance
Wenhao Liang, Chang Dong, Liangwei Zheng +2
Uncertainty is a fundamental aspect of real-world scenarios, where perfect information is rarely available. Humans naturally develop complex internal models to navigate incomplete…
Understanding Why Large Language Models Can Be Ineffective in Time Series Analysis: The Impact of Modality Alignment
Liangwei Nathan Zheng, Chang George Dong, Wei Emma Zhang +4
Large Language Models (LLMs) have demonstrated impressive performance in time series analysis and seems to understand the time temporal relationship well than traditional transform…