1 citations · 1 across the 4 of their papers we have counts for
6 papers
ExpertMuon-Compass: Alignment-Guided Step Sizes for Mixture-of-Experts Training
Omatharv Bharat Vaidya, Ashwin Vinod, Pedram Akbarian +3
Mixture-of-experts (MoE) language models send each token to a few experts, so each expert is trained on a different part of the data, and this part changes during training. With a…
When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs
Omatharv Bharat Vaidya, Connor Thomas Jerzak, Zayne Rea Sprague +2
Self-consistency assumes the most frequent answer among sampled reasoning traces is the most reliable, but this can fail in causal reasoning: samples often repeat the same confound…
Queryable LoRA: Instruction-Regularized Routing Over Shared Low-Rank Update Atoms
Omatharv Bharat Vaidya, Connor T. Jerzak, Nhat Ho +1
We present a data-adaptive method for parameter-efficient fine-tuning of large neural networks. Standard low-rank adaptation methods improve efficiency by restricting each layer up…
A Particle-based Sparse Gaussian Process Optimizer
Chandrajit Bajaj, Omatharv Bharat Vaidya, Yi Wang
Task learning in neural networks typically requires finding a globally optimal minimizer to a loss function objective. Conventional designs of swarm based optimization methods appl…
Hamiltonian Monte Carlo Particle Swarm Optimizer
Omatharv Bharat Vaidya, Rithvik Terence DSouza, Snehanshu Saha +2
We introduce the Hamiltonian Monte Carlo Particle Swarm Optimizer (HMC-PSO), an optimization algorithm that reaps the benefits of both Exponentially Averaged Momentum PSO and HMC s…
A Swarm Variant for the Schrödinger Solver
Urvil Nileshbhai Jivani, Omatharv Bharat Vaidya, Anwesh Bhattacharya +1
This paper introduces application of the Exponentially Averaged Momentum Particle Swarm Optimization (EM-PSO) as a derivative-free optimizer for Neural Networks. It adopts PSO's ma…