4 papers
Generalization Bounds for Rank-sparse Neural Networks
Antoine Ledent, Rodrigo Alves, Yunwen Lei
It has been recently observed in much of the literature that neural networks exhibit a bottleneck rank property: for larger depths, the activation and weights of neural networks tr…
Generalization and Optimization of SGD with Lookahead
Kangcheng Li, Yunwen Lei
The Lookahead optimizer enhances deep learning models by employing a dual-weight update mechanism, which has been shown to improve the performance of underlying optimizers such as…
Stability-based Generalization Analysis of Randomized Coordinate Descent for Pairwise Learning
Liang Wu, Ruixi Hu, Yunwen Lei
Pairwise learning includes various machine learning tasks, with ranking and metric learning serving as the primary representatives. While randomized coordinate descent (RCD) is pop…
Generalization Analysis for Deep Contrastive Representation Learning
Nong Minh Hieu, Antoine Ledent, Yunwen Lei +1
In this paper, we present generalization bounds for the unsupervised risk in the Deep Contrastive Representation Learning framework, which employs deep neural networks as represent…