5 papers · 1 filter
SignMuon: Communication-Efficient Distributed Muon Optimization
Neel Mishra, Jichao Jiang, Kushagara Trivedi +3
Distributed training of large neural networks is bottlenecked by full-precision gradient communication and by coordinatewise optimizers that ignore the matrix structure of weight t…
Hierarchical Sparse Plus Low Rank Compression of LLM
Pawan Kumar, Aditi Gupta
Modern large language models (LLMs) place extraordinary pressure on memory and compute budgets, making principled compression indispensable for both deployment and continued traini…
A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks
Neel Mishra, Bamdev Mishra, Pratik Jawanpuria +1
A novel first-order method is proposed for training generative adversarial networks (GANs). It modifies the Gauss-Newton method to approximate the min-max Hessian and uses the Sher…
Enhancing ML model accuracy for Digital VLSI circuits using diffusion models: A study on synthetic data generation
Prasha Srivastava, Pawan Kumar, Zia Abbas
Generative AI has seen remarkable growth over the past few years, with diffusion models being state-of-the-art for image generation. This study investigates the use of diffusion mo…
Structured Low-Rank Tensor Learning
Jayadev Naram, Tanmay Kumar Sinha, Pawan Kumar
We consider the problem of learning low-rank tensors from partial observations with structural constraints, and propose a novel factorization of such tensors, which leads to a simp…