10 citations · 20 across the 4 of their papers we have counts for
9 papers · 1 filter
Rethinking Hebbian Principle: Low-Dimensional Structural Projection for Unsupervised Learning
Shikuang Deng, Jiayuan Zhang, Yuhang Wu +2
Hebbian learning is a biological principle that intuitively describes how neurons adapt their connections through repeated stimuli. However, when applied to machine learning, it su…
Cost-Efficient Online Hyperparameter Optimization
Jingkang Wang, Mengye Ren, Ilija Bogunovic +2
Recent work on hyperparameters optimization (HPO) has shown the possibility of training certain hyperparameters together with regular parameters. However, these online HPO algorith…
Intriguing Properties of Contrastive Losses
Ting Chen, Calvin Luo, Lala Li
We study three intriguing properties of contrastive learning. First, we generalize the standard contrastive loss to a broader family of losses, and we find that various instantiati…
Understanding Why Neural Networks Generalize Well Through GSNR of Parameters
Jinlong Liu, Guoqing Jiang, Yunzhi Bai +2
As deep neural networks (DNNs) achieve tremendous success across many application domains, researchers tried to explore in many aspects on why they generalize well. In this paper,…
Differentiable Product Quantization for End-to-End Embedding Compression
Ting Chen, Lala Li, Yizhou Sun
Embedding layers are commonly used to map discrete symbols into continuous embedding vectors that reflect their semantic meanings. Despite their effectiveness, the number of parame…
Doubly Sparse: Sparse Mixture of Sparse Experts for Efficient Softmax Inference
Shun Liao, Ting Chen, Tian Lin +2
Computations for the softmax function are significantly expensive when the number of output classes is large. In this paper, we present a novel softmax inference speedup method, Do…