90 citations · 162 across the 13 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2023
Modulate Your Spectrum in Self-Supervised Learning
Xi Weng, Yunhao Ni, Tengwei Song +5
Whitening loss offers a theoretical guarantee against feature collapse in self-supervised learning (SSL) with joint embedding architectures. Typically, it involves a hard whitening…
cs.LG2017★ 90 cited
Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks
Lei Huang, Xianglong Liu, Bo Lang +3
Orthogonal matrix has shown advantages in training Recurrent Neural Networks (RNNs), but such matrix is limited to be square for the hidden-to-hidden transformation in RNNs. In thi…