11 citations · 24 across the 11 of their papers we have counts for
11 papers
NablAFx: A Framework for Differentiable Black-box and Gray-box Modeling of Audio Effects
Marco Comunità, Christian J. Steinmetz, Joshua D. Reiss
We present NablAFx, an open-source framework developed to support research in differentiable black-box and gray-box modeling of audio effects. Built in PyTorch, NablAFx offers a ve…
Differentiable Black-box and Gray-box Modeling of Nonlinear Audio Effects
Marco Comunità, Christian J. Steinmetz, Joshua D. Reiss
Audio effects are extensively used at every stage of audio and music content creation. The majority of differentiable audio effects modeling approaches fall into the black-box or g…
FolAI: Synchronized Foley Sound Generation with Semantic and Temporal Alignment
Riccardo Fosco Gramaccioni, Christian Marinoni, Emilian Postolache +4
Traditional sound design workflows rely on manual alignment of audio events to visual cues, as in Foley sound design, where everyday actions like footsteps or object interactions a…
ST-ITO: Controlling Audio Effects for Style Transfer with Inference-Time Optimization
Christian J. Steinmetz, Shubhr Singh, Marco Comunità +4
Audio production style transfer is the task of processing an input to impart stylistic elements from a reference recording. Existing approaches often train a neural network to esti…
SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyond
Marco Comunità, Zhi Zhong, Akira Takahashi +7
Recent advances in generative models that iteratively synthesize audio clips sparked great success to text-to-audio synthesis (TTA), but with the cost of slow synthesis speed and h…
SyncFusion: Multimodal Onset-synchronized Video-to-Audio Foley Synthesis
Marco Comunità, Riccardo F. Gramaccioni, Emilian Postolache +3
Sound design involves creatively selecting, recording, and editing sound effects for various media like cinema, video games, and virtual/augmented reality. One of the most time-con…
Modulation Extraction for LFO-driven Audio Effects
Christopher Mitcheltree, Christian J. Steinmetz, Marco Comunità +1
Low frequency oscillator (LFO) driven audio effects such as phaser, flanger, and chorus, modify an input signal using time-varying filters and delays, resulting in characteristic s…
Modelling black-box audio effects with time-varying feature modulation
Marco Comunità, Christian J. Steinmetz, Huy Phan +1
Deep learning approaches for black-box modelling of audio effects have shown promise, however, the majority of existing work focuses on nonlinear effects with behaviour on relative…
Neural Synthesis of Footsteps Sound Effects with Generative Adversarial Networks
Marco Comunità, Huy Phan, Joshua D. Reiss
Footsteps are among the most ubiquitous sound effects in multimedia applications. There is substantial research into understanding the acoustic features and developing synthesis mo…
Guitar Effects Recognition and Parameter Estimation with Convolutional Neural Networks
Marco Comunità, Dan Stowell, Joshua D. Reiss
Despite the popularity of guitar effects, there is very little existing research on classification and parameter estimation of specific plugins or effect units from guitar recordin…
PlugSonic: a web- and mobile-based platform for binaural audio and sonic narratives
Marco Comunità, Andrea Gerino, Veranika Lim +1
PlugSonic is a suite of web- and mobile-based applications for the curation and experience of binaural interactive soundscapes and sonic narratives. It was developed as part of the…