1.3k citations · 2.2k across the 67 of their papers we have counts for
81 papers
Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?
Zitao Song, Cedar Site Bai, Zhe Zhang +2
Training instabilities such as loss spikes are frequently the result of stochastic gradient noise. Because of rare expressions in language training data, and multiple layer composi…
Decoupling Variance and Scale-Invariant Updates in Adaptive Gradient Descent for Unified Vector and Matrix Optimization
Zitao Song, Cedar Site Bai, Zhe Zhang +2
Adaptive methods like Adam have become the standard for large-scale vector and Euclidean optimization due to their coordinate-wise adaptation with a second-orde…
Fault Oblivious Eigenvalue Solver
Jayanta Mukherjee, Xuejiao Kang, David F. Gleich +2
Eigenvalue problems serve as fundamental substrates for applications in large-scale scientific simulations and machine learning, often requiring computation on massively parallel p…
Dominant H-Eigenvectors of Tensor Kronecker Products Do Not Decouple
Ayush Kulkarni, Charles Colley, David F. Gleich
We illustrate a counterexample to an open question related to the dominant H-eigenvector of a Kronecker product of tensors. For matrices and Z-eigenvectors of tensors, the dominant…
How Fast Can Graph Computations Go on Fine-grained Parallel Architectures
Yuqing Wang, Charles Colley, Brian Wheatman +3
Large-scale graph problems are of critical and growing importance and historically parallel architectures have provided little support. In the spirit of co-design, we explore the q…
Powers of Magnetic Graph Matrix: Fourier Spectrum, Walk Compression, and Applications
Yinan Huang, David F. Gleich, Pan Li
Magnetic graphs, originally developed to model quantum systems under magnetic fields, have recently emerged as a powerful framework for analyzing complex directed networks. Existin…