24 papers
Lightweight Complementary-Cue Fusion for Robust Video Face Forgery Detection
Sunghwan Baek, Tariq Anwaar, Karanveer Singh +1
Current face video forgery detectors use wide or dual-stream backbones. We show that a single, lightweight fusion of two handcrafted cues can achieve higher accuracy with a much sm…
What and When to Learn: CURriculum Ranking Loss for Large-Scale Speaker Verification
Massa Baali, Sarthak Bisht, Rita Singh +1
Speaker verification at large scale remains an open challenge as fixed-margin losses treat all samples equally regardless of quality. We hypothesize that mislabeled or degraded sam…
VerLM: Explaining Face Verification Using Natural Language
Syed Abdul Hannan, Hazim Bukhari, Thomas Cantalapiedra +4
Face verification systems have seen substantial advancements; however, they often lack transparency in their decision-making processes. In this paper, we introduce an innovative Vi…
DELULU: Discriminative Embedding Learning Using Latent Units for Speaker-Aware Self-Trained Speech Foundational Model
Massa Baali, Rita Singh, Bhiksha Raj
Self-supervised speech models have achieved remarkable success on content-driven tasks, yet they remain limited in capturing speaker-discriminative features critical for verificati…
SVeritas: Benchmark for Robust Speaker Verification under Diverse Conditions
Massa Baali, Sarthak Bisht, Francisco Teixeira +3
Speaker verification (SV) models are increasingly integrated into security, personalization, and access control systems, yet their robustness to many real-world challenges remains…
OleSpeech-IV: A Large-Scale Multispeaker and Multilingual Conversational Speech Dataset with Diverse Topics
Wei Chu, Yuanzhe Dong, Ke Tan +7
OleSpeech-IV dataset is a large-scale multispeaker and multilingual conversational speech dataset with diverse topics. The audio content comes from publicly-available English podca…
PhoniTale: Phonologically Grounded Mnemonic Generation for Typologically Distant Language Pairs
Sana Kang, Myeongseok Gwon, Su Young Kwon +4
Vocabulary acquisition poses a significant challenge for second-language (L2) learners, especially when learning typologically distant languages such as English and Korean, where p…
Deciphering GunType Hierarchy through Acoustic Analysis of Gunshot Recordings
Ankit Shah, Rita Singh, Bhiksha Raj +1
The escalating rates of gun-related violence and mass shootings represent a significant threat to public safety. Timely and accurate information for law enforcement agencies is cru…
Human Voice is Unique
Rita Singh, Bhiksha Raj
Voice is increasingly being used as a biometric entity in many applications. These range from speaker identification and verification systems to human profiling technologies that a…
CoLMbo: Speaker Language Model for Descriptive Profiling
Massa Baali, Shuo Han, Syed Abdul Hannan +5
Speaker recognition systems are often limited to classification tasks and struggle to generate detailed speaker characteristics or provide context-rich descriptions. These models p…
Mellow: a small audio language model for reasoning
Soham Deshmukh, Satvik Dixit, Rita Singh +1
Multimodal Audio-Language Models (ALMs) can understand and reason over both audio and text. Typically, reasoning performance correlates with model size, with the best results achie…
CAARMA: Class Augmentation with Adversarial Mixup Regularization
Massa Baali, Xiang Li, Hao Chen +3
Speaker verification is a typical zero-shot learning task, where inference of unseen classes is performed by comparing embeddings of test instances to known examples. The models pe…
ADIFF: Explaining audio difference using natural language
Soham Deshmukh, Shuo Han, Rita Singh +1
Understanding and explaining differences between audio recordings is crucial for fields like audio forensics, quality assessment, and audio generation. This involves identifying an…
Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models
Hanin Atwany, Abdul Waheed, Rita Singh +2
Speech foundation models trained at a massive scale, both in terms of model and data size, result in robust systems capable of performing multiple speech tasks, including automatic…
On the Robust Approximation of ASR Metrics
Abdul Waheed, Hanin Atwany, Rita Singh +1
Recent advances in speech foundation models are largely driven by scaling both model size and data, enabling them to perform a wide range of tasks, including speech recognition. Tr…
Tessellated Linear Model for Age Prediction from Voice
Dareen Alharthi, Mahsa Zamani, Bhiksha Raj +1
Voice biometric tasks, such as age estimation require modeling the often complex relationship between voice features and the biometric variable. While deep learning models can hand…
What Do Speech Foundation Models Not Learn About Speech?
Abdul Waheed, Hanin Atwany, Bhiksha Raj +1
Understanding how speech foundation models capture non-verbal cues is crucial for improving their interpretability and adaptability across diverse tasks. In our work, we analyze se…
Improving Speaker Representations Using Contrastive Losses on Multi-scale Features
Satvik Dixit, Massa Baali, Rita Singh +1
Speaker verification systems have seen significant advancements with the introduction of Multi-scale Feature Aggregation (MFA) architectures, such as MFA-Conformer and ECAPA-TDNN.…
Did You Hear That? Introducing AADG: A Framework for Generating Benchmark Data in Audio Anomaly Detection
Ksheeraja Raghavan, Samiran Gode, Ankit Shah +4
We introduce a novel, general-purpose audio generation framework specifically designed for anomaly detection and localization. Unlike existing datasets that predominantly focus on…
PDAF: A Phonetic Debiasing Attention Framework For Speaker Verification
Massa Baali, Abdulhamid Aldoobi, Hira Dhamyal +2
Speaker verification systems are crucial for authenticating identity through voice. Traditionally, these systems focus on comparing feature vectors, overlooking the speech's conten…
Audio Entailment: Assessing Deductive Reasoning for Audio Understanding
Soham Deshmukh, Shuo Han, Hazim Bukhari +4
Recent literature uses language to build foundation models for audio. These Audio-Language Models (ALMs) are trained on a vast number of audio-text pairs and show remarkable perfor…
SELM: Enhancing Speech Emotion Recognition for Out-of-Domain Scenarios
Hazim Bukhari, Soham Deshmukh, Hira Dhamyal +2
Speech Emotion Recognition (SER) has been traditionally formulated as a classification task. However, emotions are generally a spectrum whose distribution varies from situation to…
ControlVAR: Exploring Controllable Visual Autoregressive Modeling
Xiang Li, Kai Qiu, Hao Chen +4
Conditional visual generation has witnessed remarkable progress with the advent of diffusion models (DMs), especially in tasks like control-to-image generation. However, challenges…
-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations
Xiang Li, Kai Qiu, Jinglu Wang +6
Referring perception, which aims at grounding visual objects with multimodal referring guidance, is essential for bridging the gap between humans, who provide instructions, and the…