1.4k citations · 1.7k across the 24 of their papers we have counts for
27 papers
Bridging High-Quality Audio and Video via Language for Sound Effects Retrieval from Visual Queries
Julia Wilkins, Justin Salamon, Magdalena Fuentes +2
Finding the right sound effects (SFX) to match moments in a video is a difficult and time-consuming task, and relies heavily on the quality and completeness of text metadata. Retri…
Language-Guided Music Recommendation for Video via Prompt Analogies
Daniel McKee, Justin Salamon, Josef Sivic +1
We propose a method to recommend music for an input video while allowing a user to guide music selection with free-form natural language. A key challenge of this problem setting is…
Efficient Spoken Language Recognition via Multilabel Classification
Oriol Nieto, Zeyu Jin, Franck Dernoncourt +1
Spoken language recognition (SLR) is the task of automatically identifying the language present in a speech signal. Existing SLR models are either too computationally expensive or…
Conditional Generation of Audio from Video via Foley Analogies
Yuexi Du, Ziyang Chen, Justin Salamon +2
The sound effects that designers add to videos are designed to convey a particular artistic effect and, thus, may be quite different from a scene's true sound. Inspired by the chal…
Audio-Text Models Do Not Yet Leverage Natural Language
Ho-Hsiang Wu, Oriol Nieto, Juan Pablo Bello +1
Multi-modal contrastive learning techniques in the audio-text domain have quickly become a highly active area of research. Most works are evaluated with standard audio retrieval an…
Language-Guided Audio-Visual Source Separation via Trimodal Consistency
Reuben Tan, Arijit Ray, Andrea Burns +5
We propose a self-supervised approach for learning to perform audio source separation in videos based on natural language queries, using only unlabeled video and audio pairs as tra…