289 citations · 1.3k across the 22 of their papers we have counts for
22 papers
Small-scale proxies for large-scale Transformer training instabilities
Mitchell Wortsman, Peter J. Liu, Lechao Xiao +13
Teams that have trained large Transformer-based models have reported training instabilities at large scale that did not appear when training with the same hyperparameters at smalle…
Regularized Autoencoders via Relaxed Injective Probability Flow
Abhishek Kumar, Ben Poole, Kevin Murphy
Invertible flow-based generative models are an effective method for learning to generate samples, while allowing for tractable likelihood computation and inference. However, the in…
Refined -Divergence Variational Inference via Rejection Sampling
Rahul Sharma, Abhishek Kumar, Piyush Rai
We present an approximate inference method, based on a synergistic combination of Rényi -divergence variational inference (RDVI) and rejection sampling (RS). RDVI is based on mi…
Weakly Supervised Disentanglement with Guarantees
Rui Shu, Yining Chen, Abhishek Kumar +2
Learning disentangled representations that correspond to factors of variation in real-world data is critical to interpretable and human-controllable machine learning. Recently, con…
A Scale Invariant Flatness Measure for Deep Network Minima
Akshay Rangamani, Nam H. Nguyen, Abhishek Kumar +3
It has been empirically observed that the flatness of minima obtained from training deep networks seems to correlate with better generalization. However, for deep networks with pos…
Understanding Unequal Gender Classification Accuracy from Face Images
Vidya Muthukumar, Tejaswini Pedapati, Nalini Ratha +7
Recent work shows unequal performance of commercial face classification services in the gender classification task across intersectional groups defined by skin type and gender. Acc…