203 citations · 849 across the 38 of their papers we have counts for
5 papers · 2 filters
Self-Influence Guided Data Reweighting for Language Model Pre-training
Megh Thakkar, Tolga Bolukbasi, Sriram Ganapathy +3
Language Models (LMs) pre-trained with self-supervision on large text corpora have become the default starting point for developing models for various NLP tasks. Once the pre-train…
Multimodal Modeling For Spoken Language Identification
Shikhar Bharadwaj, Min Ma, Shikhar Vashishth +10
Spoken language identification refers to the task of automatically predicting the spoken language in a given utterance. Conventionally, it is modeled as a speech-based language ide…
Label Aware Speech Representation Learning For Language Identification
Shikhar Vashishth, Shikhar Bharadwaj, Sriram Ganapathy +5
Speech representation learning approaches for non-semantic tasks such as language recognition have either explored supervised embedding extraction methods using a classifier model…
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages
Sebastian Ruder, Jonathan H. Clark, Alexander Gutkin +24
Data scarcity is a crucial issue for the development of highly multilingual NLP systems. Yet for many under-represented languages (ULs) -- languages for which NLP re-search is part…
Salient Span Masking for Temporal Understanding
Jeremy R. Cole, Aditi Chaudhary, Bhuwan Dhingra +1
Salient Span Masking (SSM) has shown itself to be an effective strategy to improve closed-book question answering performance. SSM extends general masked language model pretraining…