53 citations · 315 across the 37 of their papers we have counts for
6 papers · 1 filter
Anti-Concentrated Confidence Bonuses for Scalable Exploration
Jordan T. Ash, Cyril Zhang, Surbhi Goel +2
Intrinsic rewards play a central role in handling the exploration-exploitation trade-off when designing sequential decision-making algorithms, in both foundational theory and state…
Inductive Biases and Variable Creation in Self-Attention Mechanisms
Benjamin L. Edelman, Surbhi Goel, Sham Kakade +1
Self-attention, an architectural motif designed to model long-range interactions in sequential data, has driven numerous recent breakthroughs in natural language processing and bey…
Statistical Estimation from Dependent Data
Yuval Dagan, Constantinos Daskalakis, Nishanth Dikkala +2
We consider a general statistical estimation problem wherein binary labels across different observations are not independent conditioned on their feature vectors, but dependent, ca…
Investigating the Role of Negatives in Contrastive Representation Learning
Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy +1
Noise contrastive learning is a popular technique for unsupervised representation learning. In this approach, a representation is obtained via reduction to supervised learning, whe…
Gone Fishing: Neural Active Learning with Fisher Embeddings
Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy +1
There is an increasing need for effective active learning algorithms that are compatible with deep neural networks. This paper motivates and revisits a classic, Fisher-based active…
Acceleration via Fractal Learning Rate Schedules
Naman Agarwal, Surbhi Goel, Cyril Zhang
In practical applications of iterative first-order optimization, the learning rate schedule remains notoriously difficult to understand and expensive to tune. We demonstrate the pr…