9 papers
SignMuon: Communication-Efficient Distributed Muon Optimization
Neel Mishra, Jichao Jiang, Kushagara Trivedi +3
Distributed training of large neural networks is bottlenecked by full-precision gradient communication and by coordinatewise optimizers that ignore the matrix structure of weight t…
Hierarchical Sparse Plus Low Rank Compression of LLM
Pawan Kumar, Aditi Gupta
Modern large language models (LLMs) place extraordinary pressure on memory and compute budgets, making principled compression indispensable for both deployment and continued traini…
A Fast and Efficient Modern BERT based Text-Conditioned Diffusion Model for Medical Image Segmentation
Venkata Siddharth Dhara, Pawan Kumar
In recent times, denoising diffusion probabilistic models (DPMs) have proven effective for medical image generation and denoising, and as representation learners for downstream seg…
A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks
Neel Mishra, Bamdev Mishra, Pratik Jawanpuria +1
A novel first-order method is proposed for training generative adversarial networks (GANs). It modifies the Gauss-Newton method to approximate the min-max Hessian and uses the Sher…
Enhancing ML model accuracy for Digital VLSI circuits using diffusion models: A study on synthetic data generation
Prasha Srivastava, Pawan Kumar, Zia Abbas
Generative AI has seen remarkable growth over the past few years, with diffusion models being state-of-the-art for image generation. This study investigates the use of diffusion mo…
Nonnegative Low-Rank Tensor Completion via Dual Formulation with Applications to Image and Video Completion
Tanmay Kumar Sinha, Jayadev Naram, Pawan Kumar
Recent approaches to the tensor completion problem have often overlooked the nonnegative structure of the data. We consider the problem of learning a nonnegative low-rank tensor, a…
Structured Low-Rank Tensor Learning
Jayadev Naram, Tanmay Kumar Sinha, Pawan Kumar
We consider the problem of learning low-rank tensors from partial observations with structural constraints, and propose a novel factorization of such tensors, which leads to a simp…
Riemannian Hamiltonian methods for min-max optimization on manifolds
Andi Han, Bamdev Mishra, Pratik Jawanpuria +2
In this paper, we study min-max optimization problems on Riemannian manifolds. We introduce a Riemannian Hamiltonian function, minimization of which serves as a proxy for solving t…
On Riemannian Approach for Constrained Optimization Model in Extreme Classification Problems
Jayadev Naram, Tanmay Kumar Sinha, Pawan Kumar
We propose a novel Riemannian method for solving the Extreme multi-label classification problem that exploits the geometric structure of the sparse low-dimensional local embedding…