936 citations · 2.4k across the 134 of their papers we have counts for
22 papers · 1 filter
Optimal Guarantees for Auditing Rényi Differentially Private Machine Learning
Benjamin D. Kim, Lav R. Varshney, Daniel Alabi
We study black-box auditing for machine learning algorithms that claim R \ 'enyi differential privacy (RDP) guarantees. We introduce an auditing framework, based on hypothesis test…
A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Search
Austin R. Ellis-Mohr, Anuj K. Nayak, Lav R. Varshney
Large language models (LLMs) demand considerable computational, energy, and financial resources during both training and deployment. While scaling laws for training have guided muc…
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning
Raghav Singhal, Kaustubh Ponkshe, Rohit Vartak +2
Low-Rank Adaptation (LoRA) has become ubiquitous for efficiently fine-tuning foundation models. However, federated fine-tuning using LoRA is challenging due to suboptimal updates a…
Online Reinforcement Learning with Passive Memory
Anay Pattanaik, Lav R. Varshney
This paper considers an online reinforcement learning algorithm that leverages pre-collected data (passive memory) from the environment for online interaction. We show that using p…
Efficient Model-Agnostic Multi-Group Equivariant Networks
Razan Baltaji, Sourya Basu, Lav R. Varshney
Constructing model-agnostic group equivariant networks, such as equitune (Basu et al., 2023b) and its generalizations (Kim et al., 2023), can be computationally expensive for large…
A Meta-Learning Perspective on Transformers for Causal Language Modeling
Xinbo Wu, Lav R. Varshney
The Transformer architecture has become prominent in developing large causal language models. However, mechanisms to explain its capabilities are not well understood. Focused on th…