22 citations · 47 across the 13 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024★ 6 cited
Liger Kernel: Efficient Triton Kernels for LLM Training
Pin-Lun Hsu, Yun Dai, Vignesh Kothapalli +7
Training Large Language Models (LLMs) efficiently at scale presents a formidable challenge, driven by their ever-increasing computational demands and the need for enhanced performa…
cs.LG2024
From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
Vignesh Kothapalli, Tianyu Pang, Shenyang Deng +2
Training strategies for modern deep neural networks (NNs) tend to induce a heavy-tailed (HT) empirical spectral density (ESD) in the layer weights. While previous efforts have show…
cs.LG2024
Can Kernel Methods Explain How the Data Affects Neural Collapse?
Vignesh Kothapalli, Tom Tirer
A vast amount of literature has recently focused on the "Neural Collapse" (NC) phenomenon, which emerges when training neural network (NN) classifiers beyond the zero training erro…