16 citations · 25 across the 12 of their papers we have counts for
13 papers
Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
Bhavya Vasudeva, Puneesh Deora, Alberto Bietti +2
Transformer-based language models excel at in-context learning (ICL), where they can adapt to new tasks based on contextual examples, without parameter updates. In a specific form…
How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data
Bhavya Vasudeva, Puneesh Deora, Yize Zhao +2
The growing adoption of spectrum-aware matrix-valued optimizers such as Muon and Shampoo in deep learning motivates a systematic study of their generalization properties and, in pa…
In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
Puneesh Deora, Bhavya Vasudeva, Tina Behnia +1
In-context learning (ICL) enables transformers to adapt to new tasks through contextual examples without parameter updates. While existing research has typically studied ICL in fix…
Latent Concept Disentanglement in Transformer-based Language Models
Guan Zhe Hong, Bhavya Vasudeva, Vatsal Sharan +3
When large language models (LLMs) use in-context learning (ICL) to solve a new task, they must infer latent concepts from demonstration examples. This raises the question of whethe…
The Rich and the Simple: On the Implicit Bias of Adam and SGD
Bhavya Vasudeva, Jung Whan Lee, Vatsal Sharan +1
Adam is the de facto optimization algorithm for several deep learning applications, but an understanding of its implicit bias and how it differs from other algorithms, particularly…
Transformers Learn Low Sensitivity Functions: Investigations and Implications
Bhavya Vasudeva, Deqing Fu, Tianyi Zhou +3
Transformers achieve state-of-the-art accuracy and robustness across many tasks, but an understanding of their inductive biases and how those biases differ from other neural networ…
Implicit Bias and Fast Convergence Rates for Self-attention
Bhavya Vasudeva, Puneesh Deora, Christos Thrampoulidis
We study the fundamental optimization principles of self-attention, the defining mechanism of transformers, by analyzing the implicit bias of gradient-based optimizers in training…
Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness
Bhavya Vasudeva, Kameron Shahabi, Vatsal Sharan
Neural networks (NNs) are known to exhibit simplicity bias where they tend to prefer learning 'simple' features over more 'complex' ones, even when the latter may be more informati…
LoOp: Looking for Optimal Hard Negative Embeddings for Deep Metric Learning
Bhavya Vasudeva, Puneesh Deora, Saumik Bhattacharya +2
Deep metric learning has been effectively used to learn distance metrics for different visual tasks like image retrieval, clustering, etc. In order to aid the training process, exi…
AIM 2020 Challenge on Learned Image Signal Processing Pipeline
Andrey Ignatov, Radu Timofte, Zhilu Zhang +36
This paper reviews the second AIM learned ISP challenge and provides the description of the proposed solutions and results. The participating teams were solving a real-world RAW-to…
Co-VeGAN: Complex-Valued Generative Adversarial Network for Compressive Sensing MR Image Reconstruction
Bhavya Vasudeva, Puneesh Deora, Saumik Bhattacharya +1
Compressive sensing (CS) is widely used to reduce the acquisition time of magnetic resonance imaging (MRI). Although state-of-the-art deep learning based methods have been able to…
Efficient Implementation of LMS Adaptive Filter based FECG Extraction on an FPGA
Bhavya Vasudeva, Puneesh Deora, Pyari Mohan Pradhan +1
In this paper, the field programmable gate array (FPGA) implementation of a fetal heart rate (FHR) monitoring system is presented. The system comprises of a preprocessing unit to r…
Structure Preserving Compressive Sensing MRI Reconstruction using Generative Adversarial Networks
Puneesh Deora, Bhavya Vasudeva, Saumik Bhattacharya +1
Compressive sensing magnetic resonance imaging (CS-MRI) accelerates the acquisition of MR images by breaking the Nyquist sampling limit. In this work, a novel generative adversaria…