6 citations · 27 across the 18 of their papers we have counts for
9 papers · 1 filter
From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications
Ajay Jaiswal, Yifan Wang, Lu Yin +6
Large Language Models' (LLMs) weight matrices can often be expressed in low-rank form with potential to relax memory and compute resource requirements. Unlike prior efforts that fo…
M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation
Junjie Yang, Xuxi Chen, Tianlong Chen +2
Learning to Optimize (L2O) has drawn increasing attention as it often remarkably accelerates the optimization procedure of complex tasks by ``overfitting" specific task type, leadi…
Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?
Ruisi Cai, Zhenyu Zhang, Zhangyang Wang
Given a robust model trained to be resilient to one or multiple types of distribution shifts (e.g., natural image corruptions), how is that "robustness" encoded in the model weight…
Pruning Before Training May Improve Generalization, Provably
Hongru Yang, Yingbin Liang, Xiaojie Guo +2
It has been observed in practice that applying pruning-at-initialization methods to neural networks and training the sparsified networks can not only retain the testing performance…
Density-Aware Personalized Training for Risk Prediction in Imbalanced Medical Data
Zepeng Huo, Xiaoning Qian, Shuai Huang +2
Medical events of interest, such as mortality, often happen at a low rate in electronic medical records, as most admitted patients survive. Training models with this imbalance rate…
How Robust is Your Fairness? Evaluating and Sustaining Fairness under Unseen Distribution Shifts
Haotao Wang, Junyuan Hong, Jiayu Zhou +1
Increasing concerns have been raised on deep learning fairness in recent years. Existing fairness-aware machine learning methods mainly focus on the fairness of in-distribution dat…