5 papers
Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation
Genglin Liu, Muye Zhang, Krishnamurthy Viswanathan +5
Multimodal Large Language Models (MLLMs) are increasingly deployed for nuanced content safety and moderation tasks, yet they remain vulnerable to adversarial attacks and out-of-dis…
Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Prediction
Shu-wen Yang, Byeonggeun Kim, Kuan-Po Huang +8
Autoregressive next-token prediction with the Transformer decoder has become a de facto standard in large language models (LLMs), achieving remarkable success in Natural Language P…
IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling
Kuan-Po Huang, Shu-wen Yang, Huy Phan +8
Text-to-audio generation synthesizes realistic sounds or music given a natural language prompt. Diffusion-based frameworks, including the Tango and the AudioLDM series, represent t…
Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognition
Yash Jain, David Chan, Pranav Dheram +4
Recent advances in machine learning have demonstrated that multi-modal pre-training can improve automatic speech recognition (ASR) performance compared to randomly initialized mode…
JAB: Joint Adversarial Prompting and Belief Augmentation
Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna +6
With the recent surge of language models in different applications, attention to safety and robustness of these models has gained significant importance. Here we introduce a joint…