42 citations · 92 across the 38 of their papers we have counts for
38 papers
Bridging the Age Gap: Towards Detecting Neural Audio Codec Synthesized Elderly Speech Deepfake
Orchid Chetia Phukan, Girish, Mohd Mujtaba Akhtar +1
In this study, we introduce the Elderly CodecFake Detection (ECFD) task and release the Elderly-CodecFake (ECF) dataset in English and Chinese. We show that state-of-the-art CF det…
Towards Detecting Neural Audio Codec Synthesized Heart Sounds
Girish, Orchid Chetia Phukan, Mohd Mujtaba Akhtar +3
In this paper, we introduce Synthetic Heart Sound Detection (SHAC), a task aimed at identifying phonocardiograms (PCGs) synthesized using neural audio codecs (NACs). To facilitate…
Bridging the SEA Gap: An Initial Benchmark for Neural Audio Codec-Synthesized Speech Deepfakes in South-East Asian Languages
Orchid Chetia Phukan, Girish, Mohd Mujtaba Akhtar +1
Codecfakes (CFs) are a type of speech deepfakes generated through Audio Language Models (ALMs), with Neural Audio Codecs (NACs) forming the core mechanism for speech encoding and g…
Impact Analysis of Speech Representation Learning Models for Acoustic Side-Channel Attack
Nitin Choudhury, Bikrant Bikram Pratap Maurya, Arun Balaji Buduru +2
Acoustic side-channel attacks (ASCA) on keyboards have gained increasing attention, yet impact of speech representation learning models in ASCA remains unexplored. Addressing this,…
RoboKA: KAN Informed Multimodal Learning for RoboCall Surveillance System
Nitin Choudhury, Nikhil Kumar, Aditya Kumar Sinha +5
Wide exploration on robocall surveillance research is hindered due to limited access to public datasets, due to privacy concerns. In this work, we first curate Robo-SAr, a syntheti…
Indic-CodecFake meets SATYAM: Towards Detecting Neural Audio Codec Synthesized Speech Deepfakes in Indic Languages
Girish, Mohd Mujtaba Akhtar, Orchid Chetia Phukan +1
The rapid advancement of Audio Large Language Models (ALMs), driven by Neural Audio Codecs (NACs), has led to the emergence of highly realistic speech deepfakes, commonly referred…
FOCA: Multimodal Malware Classification via Hyperbolic Cross-Attention
Nitin Choudhury, Bikrant Bikram Pratap Maurya, Orchid Chetia Phukan +1
In this work, we introduce FOCA, a novel multimodal framework for malware classification that jointly leverages audio and visual modalities. Unlike conventional Euclidean-based fus…
Are Multimodal Foundation Models All That Is Needed for Emofake Detection?
Mohd Mujtaba Akhtar, Girish, Orchid Chetia Phukan +5
In this work, we investigate multimodal foundation models (MFMs) for EmoFake detection (EFD) and hypothesize that they will outperform audio foundation models (AFMs). MFMs due to t…
SNIFR : Boosting Fine-Grained Child Harmful Content Detection Through Audio-Visual Alignment with Cascaded Cross-Transformer
Orchid Chetia Phukan, Mohd Mujtaba Akhtar, Girish +8
As video-sharing platforms have grown over the past decade, child viewership has surged, increasing the need for precise detection of harmful content like violence or explicit scen…
Are Mamba-based Audio Foundation Models the Best Fit for Non-Verbal Emotion Recognition?
Mohd Mujtaba Akhtar, Orchid Chetia Phukan, Girish +5
In this work, we focus on non-verbal vocal sounds emotion recognition (NVER). We investigate mamba-based audio foundation models (MAFMs) for the first time for NVER and hypothesize…
Source Tracing of Synthetic Speech Systems Through Paralinguistic Pre-Trained Representations
Girish, Mohd Mujtaba Akhtar, Orchid Chetia Phukan +5
In this work, we focus on source tracing of synthetic speech generation systems (STSGS). Each source embeds distinctive paralinguistic features--such as pitch, tone, rhythm, and in…
Investigating Prosodic Signatures via Speech Pre-Trained Models for Audio Deepfake Source Attribution
Orchid Chetia Phukan, Drishti Singh, Swarup Ranjan Behera +2
In this work, we investigate various state-of-the-art (SOTA) speech pre-trained models (PTMs) for their capability to capture prosodic signatures of the generative sources for audi…
Multi-View Multi-Task Modeling with Speech Foundation Models for Speech Forensic Tasks
Orchid Chetia Phukan, Devyani Koshal, Swarup Ranjan Behera +2
Speech forensic tasks (SFTs), such as automatic speaker recognition (ASR), speech emotion recognition (SER), gender recognition (GR), and age estimation (AE), find use in different…
SeQuiFi: Mitigating Catastrophic Forgetting in Speech Emotion Recognition with Sequential Class-Finetuning
Sarthak Jain, Orchid Chetia Phukan, Swarup Ranjan Behera +2
In this work, we introduce SeQuiFi, a novel approach for mitigating catastrophic forgetting (CF) in speech emotion recognition (SER). SeQuiFi adopts a sequential class-finetuning s…
Avengers Assemble: Amalgamation of Non-Semantic Features for Depression Detection
Orchid Chetia Phukan, Swarup Ranjan Behera, Shubham Singh +5
In this study, we address the challenge of depression detection from speech, focusing on the potential of non-semantic features (NSFs) to capture subtle markers of depression. Whil…
Strong Alone, Stronger Together: Synergizing Modality-Binding Foundation Models with Optimal Transport for Non-Verbal Emotion Recognition
Orchid Chetia Phukan, Mohd Mujtaba Akhtar, Girish +5
In this study, we investigate multimodal foundation models (MFMs) for emotion recognition from non-verbal sounds. We hypothesize that MFMs, with their joint pre-training across mul…
Are Music Foundation Models Better at Singing Voice Deepfake Detection? Far-Better Fuse them with Speech Foundation Models
Orchid Chetia Phukan, Sarthak Jain, Swarup Ranjan Behera +3
In this study, for the first time, we extensively investigate whether music foundation models (MFMs) or speech foundation models (SFMs) work better for singing voice deepfake detec…
ASGIR: Audio Spectrogram Transformer Guided Classification And Information Retrieval For Birds
Yashwardhan Chaudhuri, Paridhi Mundra, Arnesh Batra +2
Recognition and interpretation of bird vocalizations are pivotal in ornithological research and ecological conservation efforts due to their significance in understanding avian beh…
VoxMed: One-Step Respiratory Disease Classifier using Digital Stethoscope Sounds
Paridhi Mundra, Manik Sharma, Yashwardhan Chaudhuri +2
As respiratory illnesses become more common, it is crucial to quickly and accurately detect them to improve patient care. There is a need for improved diagnostic methods for immedi…
Modality-Order Matters! A Novel Hierarchical Feature Fusion Method for CoSAm: A Code-Switched Autism Corpus
Mohd Mujtaba Akhtar, Girish, Muskaan Singh +1
Autism Spectrum Disorder (ASD) is a complex neuro-developmental challenge, presenting a spectrum of difficulties in social interaction, communication, and the expression of repetit…
AVR: Synergizing Foundation Models for Audio-Visual Humor Detection
Sarthak Sharma, Orchid Chetia Phukan, Drishti Singh +2
In this work, we present, AVR application for audio-visual humor detection. While humor detection has traditionally centered around textual analysis, recent advancements have spotl…
Towards Multilingual Audio-Visual Question Answering
Orchid Chetia Phukan, Priyabrata Mallick, Swarup Ranjan Behera +3
In this paper, we work towards extending Audio-Visual Question Answering (AVQA) to multilingual settings. Existing AVQA research has predominantly revolved around English and repli…
The Reasonable Effectiveness of Speaker Embeddings for Violence Detection
Sarthak Jain, Orchid Chetia Phukan, Arun Balaji Buduru +1
In this paper, we focus on audio violence detection (AVD). AVD is necessary for several reasons, especially in the context of maintaining safety, preventing harm, and ensuring secu…
PERSONA: An Application for Emotion Recognition, Gender Recognition and Age Estimation
Devyani Koshal, Orchid Chetia Phukan, Sarthak Jain +2
Emotion Recognition (ER), Gender Recognition (GR), and Age Estimation (AE) constitute paralinguistic tasks that rely not on the spoken content but primarily on speech characteristi…