4 papers
MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling
Hsing-Huan Chung, Shijun Li, Yoav Wald +3
Multimodal irregular time series (MITS) consist of asynchronous and irregularly sampled observations from heterogeneous numerical and textual channels. In healthcare, for example,…
FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning
Xing Han, Shravan Chaudhari, Tanvi Ranade +2
Real-world model deployment across multiple domains requires multimodal models to operate under two complementary regimes: (1) multi-task pretraining, tasks are co-available at des…
On the Invariance and Generality of Neural Scaling Laws
Xing Han, Ziyin Liu, Suchi Saria +1
Neural scaling laws establish a predictable relationship between model performance and data or compute, offering crucial guidance for resource allocation in new domains and tasks.…
MINT: Multimodal Instruction Tuning with Multimodal Interaction Grouping
Xiaojun Shan, Qi Cao, Xing Han +2
Recent advances in multimodal foundation models have achieved state-of-the-art performance across a range of tasks. These breakthroughs are largely driven by new pre-training parad…