2 citations · 2 across the 4 of their papers we have counts for
6 papers
What Happens During the Loss Plateau? Understanding Abrupt Learning in Transformers
Pulkit Gopalani, Wei Hu
Training Transformers on algorithmic tasks frequently demonstrates an intriguing abrupt learning phenomenon: an extended performance plateau followed by a sudden, sharp improvement…
Abrupt Learning in Transformers: A Case Study on Matrix Completion
Pulkit Gopalani, Ekdeep Singh Lubana, Wei Hu
Recent analysis on the training dynamics of Transformers has unveiled an interesting characteristic: the training loss plateaus for a significant number of training steps, and then…
Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets
Pulkit Gopalani, Samyak Jha, Anirbit Mukherjee
In this note, we demonstrate a first-of-its-kind provable convergence of SGD to the global minima of appropriately regularized logistic empirical risk of depth nets -- for arbi…
Global Convergence of SGD On Two Layer Neural Nets
Pulkit Gopalani, Anirbit Mukherjee
In this note, we consider appropriately regularized empirical risk of depth nets with any number of gates and show bounds on how the empirical loss evolves for SGD ite…
Towards Size-Independent Generalization Bounds for Deep Operator Nets
Pulkit Gopalani, Sayar Karmakar, Dibyakanti Kumar +1
In recent times machine learning methods have made significant advances in becoming a useful tool for analyzing physical systems. A particularly active area in this theme has been…
On Adversarial Robustness: A Neural Architecture Search perspective
Chaitanya Devaguptapu, Devansh Agarwal, Gaurav Mittal +2
Adversarial robustness of deep learning models has gained much traction in the last few years. Various attacks and defenses are proposed to improve the adversarial robustness of mo…