5 papers
Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed
Haokun Lin, Kaijie Zhu, Haobo Xu +4
Small Language Models (SLMs) have emerged as a more efficient alternative to traditional Large Language Models (LLMs), offering promising potential in resource-constrained scenario…
DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization
Haokun Lin, Xinle Jia, Haobo Xu +7
The MXFP4 microscaling format, which partitions tensors into blocks of 32 elements sharing an E8M0 scaling factor, has emerged as a promising substrate for efficient LLM inference,…
Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs
Haokun Lin, Haobo Xu, Yichen Wu +6
Recent advances in diffusion large language models (dLLMs) have introduced a promising alternative to autoregressive (AR) LLMs for natural language generation tasks, leveraging ful…
Resolving Conflicts in Lifelong Learning via Aligning Updates in Subspaces
Yueer Zhou, Yichen Wu, Ying Wei
Low-Rank Adaptation (LoRA) enables efficient Continual Learning but often suffers from catastrophic forgetting due to destructive interference between tasks. Our analysis reveals t…
SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning
Yichen Wu, Hongming Piao, Long-Kai Huang +6
Continual Learning (CL) with foundation models has recently emerged as a promising paradigm to exploit abundant knowledge acquired during pre-training for tackling sequential tasks…