190 citations · 1.1k across the 65 of their papers we have counts for
58 papers · 1 filter
Memory Efficient Optimizers with 4-bit States
Bingrui Li, Jianfei Chen, Jun Zhu
Optimizer states are a major source of memory consumption for training neural networks, limiting the maximum trainable model within given memory budget. Compressing the optimizer s…
Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence
Liyuan Wang, Xingxing Zhang, Qian Li +4
Continual learning aims to empower artificial intelligence (AI) with strong adaptability to the real world. For this purpose, a desirable solution should properly balance memory st…
Towards Accelerated Model Training via Bayesian Data Selection
Zhijie Deng, Peng Cui, Jun Zhu
Mislabeled, duplicated, or biased data in real-world scenarios can lead to prolonged training and even hinder model convergence. Traditional solutions prioritizing easy or hard sam…
MultiAdam: Parameter-wise Scale-invariant Optimizer for Multiscale Training of Physics-informed Neural Networks
Jiachen Yao, Chang Su, Zhongkai Hao +3
Physics-informed Neural Networks (PINNs) have recently achieved remarkable progress in solving Partial Differential Equations (PDEs) in various fields by minimizing a weighted sum…
PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs
Zhongkai Hao, Jiachen Yao, Chang Su +8
While significant progress has been made on Physics-Informed Neural Networks (PINNs), a comprehensive comparison of these methods across a wide range of Partial Differential Equati…
Deep Ensemble as a Gaussian Process Approximate Posterior
Zhijie Deng, Feng Zhou, Jianfei Chen +2
Deep Ensemble (DE) is an effective alternative to Bayesian neural networks for uncertainty quantification in deep learning. The uncertainty of DE is usually conveyed by the functio…