4 papers · 1 filter
Faster Query-Key Learning Sharpens Attention in Self-Attention Models
Rahul Vashisht, Harish G. Ramaswamy
A standard self-attention layer consists of two interacting circuits: the query-key circuit that governs attention allocation, and the output-value circuit that maps attended repre…
Impact of Label Noise on Learning Complex Features
Rahul Vashisht, P. Krishna Kumar, Harsha Vardhan Govind +1
Neural networks trained with stochastic gradient descent exhibit an inductive bias towards simpler decision boundaries, typically converging to a narrow family of functions, and of…
Using noise resilience for ranking generalization of deep neural networks
Depen Morwani, Rahul Vashisht, Harish G. Ramaswamy
Recent papers have shown that sufficiently overparameterized neural networks can perfectly fit even random labels. Thus, it is crucial to understand the underlying reason behind th…
Structural Health Monitoring of Cantilever Beam, a Case Study -- Using Bayesian Neural Network AND Deep Learning
Rahul Vashisht, H. Viji, T. Sundararajan +2
The advancement of machine learning algorithms has opened a wide scope for vibration-based SHM (Structural Health Monitoring). Vibration-based SHM is based on the fact that damage…