4 papers
Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks
Aditi Gupta, Neel Mishra, Kushagra Trivedi +1
How should we evaluate generation systems that combine autoregressive (AR) and diffusion decoding? We study this question through Speculative Refinement (SpecRef), a training-free…
SignMuon: Communication-Efficient Distributed Muon Optimization
Neel Mishra, Jichao Jiang, Kushagara Trivedi +3
Distributed training of large neural networks is bottlenecked by full-precision gradient communication and by coordinatewise optimizers that ignore the matrix structure of weight t…
A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks
Neel Mishra, Bamdev Mishra, Pratik Jawanpuria +1
A novel first-order method is proposed for training generative adversarial networks (GANs). It modifies the Gauss-Newton method to approximate the min-max Hessian and uses the Sher…
Angle based dynamic learning rate for gradient descent
Neel Mishra, Pawan Kumar
In our work, we propose a novel yet simple approach to obtain an adaptive learning rate for gradient-based descent methods on classification tasks. Instead of the traditional appro…