12 citations · 17 across the 6 of their papers we have counts for
6 papers
Faster Query-Key Learning Sharpens Attention in Self-Attention Models
Rahul Vashisht, Harish G. Ramaswamy
A standard self-attention layer consists of two interacting circuits: the query-key circuit that governs attention allocation, and the output-value circuit that maps attended repre…
Impact of Label Noise on Learning Complex Features
Rahul Vashisht, P. Krishna Kumar, Harsha Vardhan Govind +1
Neural networks trained with stochastic gradient descent exhibit an inductive bias towards simpler decision boundaries, typically converging to a narrow family of functions, and of…
On the Learning Dynamics of Attention Networks
Rahul Vashisht, Harish G. Ramaswamy
Attention models are typically learned by optimizing one of three standard loss functions that are variously called -- soft attention, hard attention, and latent variable marginal…
On the Interpretability of Attention Networks
Lakshmi Narayan Pandey, Rahul Vashisht, Harish G. Ramaswamy
Attention mechanisms form a core component of several successful deep learning architectures, and are based on one key idea: ''The output depends only on a small (but unknown) segm…
Using noise resilience for ranking generalization of deep neural networks
Depen Morwani, Rahul Vashisht, Harish G. Ramaswamy
Recent papers have shown that sufficiently overparameterized neural networks can perfectly fit even random labels. Thus, it is crucial to understand the underlying reason behind th…
Structural Health Monitoring of Cantilever Beam, a Case Study -- Using Bayesian Neural Network AND Deep Learning
Rahul Vashisht, H. Viji, T. Sundararajan +2
The advancement of machine learning algorithms has opened a wide scope for vibration-based SHM (Structural Health Monitoring). Vibration-based SHM is based on the fact that damage…