7 papers
Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization
Fei Wang, Chao Xue, Taoran Liu +3
Mixed-precision quantization (MPQ) has become a key technique for deploying large language models under stringent memory and compute constraints. We first identify a phenomenon tha…
From Consistency to Complementarity: Aligned and Disentangled Multi-modal Learning for Time Series Understanding and Reasoning
Hang Ni, Weijia Zhang, Fei Wang +2
Advances in multi-modal large language models (MLLMs) have inspired time series understanding and reasoning tasks, that enable natural language querying over time series, producing…
DropoutTS: Sample-Adaptive Dropout for Robust Time Series Forecasting
Siru Zhong, Yiqiu Liu, Zhiqing Cui +4
Deep time series models are vulnerable to noisy data ubiquitous in real-world applications. Existing robustness strategies either prune data or rely on costly prior quantification,…
Layer as Puzzle Pieces: Compressing Large Language Models through Layer Concatenation
Fei Wang, Li Shen, Liang Ding +3
Large Language Models excel at natural language processing tasks, but their massive size leads to high computational and storage demands. Recent works have sought to reduce their m…
DynScaling: Efficient Verifier-free Inference Scaling via Dynamic and Integrated Sampling
Fei Wang, Xingchen Wan, Ruoxi Sun +2
Inference-time scaling has proven effective in boosting large language model (LLM) performance through increased test-time computation. Yet, its practical application is often hind…
How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook
Haoxin Liu, Harshavardhan Kamarthi, Zhiyuan Zhao +6
Time series analysis (TSA) is a longstanding research topic in the data mining community and has wide real-world significance. Compared to "richer" modalities such as language and…
Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
Fei Wang, Li Shen, Liang Ding +3
Fine-tuning is powerful for adapting large language models to downstream tasks, but it often results in huge memory usages. A promising approach to mitigate this is using Zeroth-Or…