187 citations · 698 across the 55 of their papers we have counts for
12 papers · 1 filter
Error-aware Quantization through Noise Tempering
Zheng Wang, Juncheng B Li, Shuhui Qu +2
Quantization has become a predominant approach for model compression, enabling deployment of large models trained on GPUs onto smaller form-factor devices for inference. Quantizati…
Normalized Contrastive Learning for Text-Video Retrieval
Yookoon Park, Mahmoud Azab, Bo Xiong +4
Cross-modal contrastive learning has led the recent advances in multimodal retrieval with its simplicity and effectiveness. In this work, however, we reveal that cross-modal contra…
Token-level Sequence Labeling for Spoken Language Understanding using Compositional End-to-End Models
Siddhant Arora, Siddharth Dalmia, Brian Yan +3
End-to-end spoken language understanding (SLU) systems are gaining popularity over cascaded approaches due to their simplicity and ability to avoid error propagation. However, thes…
SQuAT: Sharpness- and Quantization-Aware Training for BERT
Zheng Wang, Juncheng B Li, Shuhui Qu +2
Quantization is an effective technique to reduce memory footprint, inference latency, and power consumption of deep learning models. However, existing quantization methods suffer f…
CTC Alignments Improve Autoregressive Translation
Brian Yan, Siddharth Dalmia, Yosuke Higuchi +4
Connectionist Temporal Classification (CTC) is a widely used approach for automatic speech recognition (ASR) that performs conditionally independent monotonic alignment. However fo…
ASR2K: Speech Recognition for Around 2000 Languages without Audio
Xinjian Li, Florian Metze, David R Mortensen +2
Most recent speech recognition models rely on large supervised datasets, which are unavailable for many low-resource languages. In this work, we present a speech recognition pipeli…
Masked Autoencoders that Listen
Po-Yao Huang, Hu Xu, Juncheng Li +5
This paper studies a simple extension of image-based Masked Autoencoders (MAE) to self-supervised representation learning from audio spectrograms. Following the Transformer encoder…
LegoNN: Building Modular Encoder-Decoder Models
Siddharth Dalmia, Dmytro Okhonko, Mike Lewis +5
State-of-the-art encoder-decoder models (e.g. for machine translation (MT) or automatic speech recognition (ASR)) are constructed and trained end-to-end as an atomic unit. No compo…
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization
Shruti Palaskar, Akshita Bhagia, Yonatan Bisk +3
Combining the visual modality with pretrained language models has been surprisingly effective for simple descriptive tasks such as image captioning. More general text generation ho…
Robustness of Neural Architectures for Audio Event Detection
Juncheng B Li, Zheng Wang, Shuhui Qu +1
Traditionally, in Audio Recognition pipeline, noise is suppressed by the "frontend", relying on preprocessing techniques such as speech enhancement. However, it is not guaranteed t…
On Adversarial Robustness of Large-scale Audio Visual Learning
Juncheng B Li, Shuhui Qu, Xinjian Li +2
As audio-visual systems are being deployed for safety-critical tasks such as surveillance and malicious content filtering, their robustness remains an under-studied area. Existing…
AudioTagging Done Right: 2nd comparison of deep learning methods for environmental sound classification
Juncheng B Li, Shuhui Qu, Po-Yao Huang +1
After its sweeping success in vision and language tasks, pure attention-based neural architectures (e.g. DeiT) are emerging to the top of audio tagging (AT) leaderboards, which see…