3 citations · 6 across the 14 of their papers we have counts for
16 papers
The Tokens Strike Back: Safety-Aware Denoising with Directional Embedding-based Revision
Amman Yusuf, Jiwoo Shin, Ruizhi Deng +3
The Safety-Aware Denoiser (SAD) is a recent, training-free, safety-guidance framework for text diffusion models. During inference, SAD steers text samples toward safe regions of th…
Improved Monitoring of Honey bee Colony Strength via Audio IoT Sensors, Modulation Tensorgrams and Recurrent Neural Networks
Mahsa Abdollahi, Yi Zhu, Heitor R. Guimarães +4
Honey bees (Apis mellifera) play a crucial role in agriculture and ecosystem stability as key pollinators of crops and wild plants. As such, monitoring hive strength remotely with…
BioME: A Resource-Efficient Bioacoustic Foundational Model for IoT Applications
Heitor R. Guimarães, Abhishek Tiwari, Mahsa Abdollahi +2
Passive acoustic monitoring has become a key strategy in biodiversity assessment, conservation, and behavioral ecology, especially as Internet-of-Things (IoT) devices enable contin…
AUDDT: A Unified Benchmark Toolkit for Audio and Speech Deepfake Detectors
Yi Zhu, Heitor R. Guimarães, Arthur Pimentel +1
With the prevalence of artificial intelligence (AI)-generated content, such as audio deepfakes, a large body of recent work has focused on developing deepfake detection techniques.…
Improving Resource-Efficient Speech Enhancement via Neural Differentiable DSP Vocoder Refinement
Heitor R. Guimarães, Ke Tan, Juan Azcarreta +4
Deploying speech enhancement (SE) systems in wearable devices, such as smart glasses, is challenging due to the limited computational resources on the device. Although deep learnin…
DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformers
Heitor R. Guimarães, Jiaqi Su, Rithesh Kumar +2
Real-world speech recordings suffer from degradations such as background noise and reverberation. Speech enhancement aims to mitigate these issues by generating clean high-fidelity…
Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks
Chien-yu Huang, Wei-Chih Chen, Shu-wen Yang +77
Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spo…
UrBAN: Urban Beehive Acoustics and PheNotyping Dataset
Mahsa Abdollahi, Yi Zhu, Heitor R. Guimarães +4
In this paper, we present a multimodal dataset obtained from a honey bee colony in Montréal, Quebec, Canada, spanning the years of 2021 to 2022. This apiary comprised 10 beehives,…
An Efficient End-to-End Approach to Noise Invariant Speech Features via Multi-Task Learning
Heitor R. Guimarães, Arthur Pimentel, Anderson R. Avila +3
Self-supervised speech representation learning enables the extraction of meaningful features from raw waveforms. These features can then be efficiently used across multiple downstr…
MSPB: a longitudinal multi-sensor dataset with phenotypic trait measurements from honey bees
Yi Zhu, Mahsa Abdollahi, Ségolène Maucourt +4
We present a longitudinal multi-sensor dataset collected from honey bee colonies (Apis mellifera) with rich phenotypic measurements. Data were continuously collected between May-20…
On the Impact of Quantization and Pruning of Self-Supervised Speech Models for Downstream Speech Recognition Tasks "In-the-Wild''
Arthur Pimentel, Heitor Guimarães, Anderson R. Avila +2
Recent advances with self-supervised learning have allowed speech recognition systems to achieve state-of-the-art (SOTA) word error rates (WER) while requiring only a fraction of t…
VIC-KD: Variance-Invariance-Covariance Knowledge Distillation to Make Keyword Spotting More Robust Against Adversarial Attacks
Heitor R. Guimarães, Arthur Pimentel, Anderson Avila +1
Keyword spotting (KWS) refers to the task of identifying a set of predefined words in audio streams. With the advances seen recently with deep neural networks, it has become a popu…
On the Transferability of Whisper-based Representations for "In-the-Wild" Cross-Task Downstream Speech Applications
Vamsikrishna Chemudupati, Marzieh Tahaei, Heitor Guimaraes +5
Large self-supervised pre-trained speech models have achieved remarkable success across various speech-processing tasks. The self-supervised training of these models leads to unive…
An Exploration into the Performance of Unsupervised Cross-Task Speech Representations for "In the Wild'' Edge Applications
Heitor Guimarães, Arthur Pimentel, Anderson Avila +2
Unsupervised speech models are becoming ubiquitous in the speech and machine learning communities. Upstream models are responsible for learning meaningful representations from raw…
RobustDistiller: Compressing Universal Speech Representations for Enhanced Environment Robustness
Heitor R. Guimarães, Arthur Pimentel, Anderson R. Avila +3
Self-supervised speech pre-training enables deep neural network models to capture meaningful and disentangled factors from raw waveform signals. The learned universal speech repres…
Improving the Robustness of DistilHuBERT to Unseen Noisy Conditions via Data Augmentation, Curriculum Learning, and Multi-Task Enhancement
Heitor R. Guimarães, Arthur Pimentel, Anderson R. Avila +2
Self-supervised speech representation learning aims to extract meaningful factors from the speech signal that can later be used across different downstream tasks, such as speech an…