61 citations · 112 across the 13 of their papers we have counts for
5 papers · 1 filter
OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages
Prem Selvaraj, Gokul NC, Pratyush Kumar +1
AI technologies for Natural Languages have made tremendous progress recently. However, commensurate progress has not been made on Sign Languages, in particular, in recognizing sign…
On the Prunability of Attention Heads in Multilingual BERT
Aakriti Budhraja, Madhura Pande, Pratyush Kumar +1
Large multilingual models, such as mBERT, have shown promise in crosslingual transfer. In this work, we employ pruning to quantify the robustness and interpret layer-wise importanc…
The heads hypothesis: A unifying statistical approach towards understanding multi-headed attention in BERT
Madhura Pande, Aakriti Budhraja, Preksha Nema +2
Multi-headed attention heads are a mainstay in transformer-based models. Different methods have been proposed to classify the role of each attention head based on the relations bet…
On the Importance of Local Information in Transformer Based Models
Madhura Pande, Aakriti Budhraja, Preksha Nema +2
The self-attention module is a key component of Transformer-based models, wherein each token pays attention to every other token. Recent studies have shown that these heads exhibit…
AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages
Anoop Kunchukuttan, Divyanshu Kakwani, Satish Golla +4
We present the IndicNLP corpus, a large-scale, general-domain corpus containing 2.7 billion words for 10 Indian languages from two language families. We share pre-trained word embe…