3 citations · 8 across the 11 of their papers we have counts for
11 papers
Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data
Yifan Peng, Jinchuan Tian, Brian Yan +13
Pre-training speech models on large volumes of data has achieved remarkable success. OpenAI Whisper is a multilingual multitask model trained on 680k hours of supervised speech dat…
Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study
Xuankai Chang, Brian Yan, Kwanghee Choi +14
Speech signals, typically sampled at rates in the tens of thousands per second, contain redundancies, evoking inefficiencies in sequence modeling. High-dimensional speech features…
Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
Chien-yu Huang, Ke-Han Lu, Shih-Heng Wang +12
Text language models have shown remarkable zero-shot capability in generalizing to unseen tasks when provided with well-formulated instructions. However, existing studies in speech…
BASS: Block-wise Adaptation for Speech Summarization
Roshan Sharma, Kenneth Zheng, Siddhant Arora +3
End-to-end speech summarization has been shown to improve performance over cascade baselines. However, such models are difficult to train on very large inputs (dozens of minutes or…
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks
Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin +7
Spoken language understanding (SLU) tasks have been studied for many decades in the speech research community, but have not received as much attention as lower-level tasks like spe…
Egocentric Audio-Visual Noise Suppression
Roshan Sharma, Weipeng He, Ju Lin +3
This paper studies audio-visual noise suppression for egocentric videos -- where the speaker is not captured in the video. Instead, potential noise sources are visible on screen wi…
Unifying the Discrete and Continuous Emotion labels for Speech Emotion Recognition
Roshan Sharma, Hira Dhamyal, Bhiksha Raj +1
Traditionally, in paralinguistic analysis for emotion detection from speech, emotions have been identified with discrete or dimensional (continuous-valued) labels. Accordingly, mod…
XNOR-FORMER: Learning Accurate Approximations in Long Speech Transformers
Roshan Sharma, Bhiksha Raj
Transformers are among the state of the art for many tasks in speech, vision, and natural language processing, among others. Self-attentions, which are crucial contributors to this…
Self-supervision and Learnable STRFs for Age, Emotion, and Country Prediction
Roshan Sharma, Tyler Vuong, Mark Lindsey +3
This work presents a multitask approach to the simultaneous estimation of age, country of origin, and emotion given vocal burst audio for the 2022 ICML Expressive Vocalizations Cha…
Speech Summarization using Restricted Self-Attention
Roshan Sharma, Shruti Palaskar, Alan W Black +1
Speech summarization is typically performed by using a cascade of speech recognition and text summarization models. End-to-end modeling of speech summarization models is challengin…
A Summary of the First Workshop on Language Technology for Language Documentation and Revitalization
Graham Neubig, Shruti Rijhwani, Alexis Palmer +21
Despite recent advances in natural language processing and other language technology, the application of such technology to language documentation and conservation has been limited…