3 papers
cs.CL2025
FlowNIB: An Information Bottleneck Analysis of Bidirectional vs. Unidirectional Language Models
Md Kowsher, Nusrat Jahan Prottasha, Shiyun Xu +4
Bidirectional language models have better context understanding and perform better than unidirectional models on natural language understanding tasks, yet the theoretical reasons b…
cs.LG2025
Adaptive parameter-efficient fine-tuning via Hessian-informed subset selection
Shiyun Xu, Zhiqi Bu
Parameter-efficient fine-tuning (PEFT) is a highly effective approach for adapting large pre-trained models to downstream tasks with minimal computational overhead. At the core, PE…
cs.LG2025
A Hessian-informed hyperparameter optimization for differential learning rate
Shiyun Xu, Zhiqi Bu, Yiliang Zhang +1
Differential learning rate (DLR), a technique that applies different learning rates to different model parameters, has been widely used in deep learning and achieved empirical succ…