36 citations · 74 across the 20 of their papers we have counts for
20 papers
NablAFx: A Framework for Differentiable Black-box and Gray-box Modeling of Audio Effects
Marco Comunità, Christian J. Steinmetz, Joshua D. Reiss
We present NablAFx, an open-source framework developed to support research in differentiable black-box and gray-box modeling of audio effects. Built in PyTorch, NablAFx offers a ve…
Differentiable Black-box and Gray-box Modeling of Nonlinear Audio Effects
Marco Comunità, Christian J. Steinmetz, Joshua D. Reiss
Audio effects are extensively used at every stage of audio and music content creation. The majority of differentiable audio effects modeling approaches fall into the black-box or g…
ST-ITO: Controlling Audio Effects for Style Transfer with Inference-Time Optimization
Christian J. Steinmetz, Shubhr Singh, Marco Comunità +4
Audio production style transfer is the task of processing an input to impart stylistic elements from a reference recording. Existing approaches often train a neural network to esti…
Diff-MST: Differentiable Mixing Style Transfer
Soumya Sai Vanka, Christian Steinmetz, Jean-Baptiste Rolland +2
Mixing style transfer automates the generation of a multitrack mix for a given set of tracks by inferring production attributes from a reference song. However, existing systems for…
Modeling Analog Dynamic Range Compressors using Deep Learning and State-space Models
Hanzhi Yin, Gang Cheng, Christian J. Steinmetz +3
We describe a novel approach for developing realistic digital models of dynamic range compressors for digital audio production by analyzing their analog prototypes. While realistic…
ATGNN: Audio Tagging Graph Neural Network
Shubhr Singh, Christian J. Steinmetz, Emmanouil Benetos +2
Deep learning models such as CNNs and Transformers have achieved impressive performance for end-to-end audio tagging. Recent works have shown that despite stacking multiple layers,…
High-Fidelity Noise Reduction with Differentiable Signal Processing
Christian J. Steinmetz, Thomas Walther, Joshua D. Reiss
Noise reduction techniques based on deep learning have demonstrated impressive performance in enhancing the overall quality of recorded speech. While these approaches are highly pe…
General Purpose Audio Effect Removal
Matthew Rice, Christian J. Steinmetz, George Fazekas +1
Although the design and application of audio effects is well understood, the inverse problem of removing these effects is significantly more challenging and far less studied. Recen…
Modulation Extraction for LFO-driven Audio Effects
Christopher Mitcheltree, Christian J. Steinmetz, Marco Comunità +1
Low frequency oscillator (LFO) driven audio effects such as phaser, flanger, and chorus, modify an input signal using time-varying filters and delays, resulting in characteristic s…
Leveraging Neural Representations for Audio Manipulation
Scott H. Hawley, Christian J. Steinmetz
We investigate applying audio manipulations using pretrained neural network-based autoencoders as an alternative to traditional signal processing methods, since the former may prov…
Modelling black-box audio effects with time-varying feature modulation
Marco Comunità, Christian J. Steinmetz, Huy Phan +1
Deep learning approaches for black-box modelling of audio effects have shown promise, however, the majority of existing work focuses on nonlinear effects with behaviour on relative…
Style Transfer of Audio Effects with Differentiable Signal Processing
Christian J. Steinmetz, Nicholas J. Bryan, Joshua D. Reiss
We present a framework that can impose the audio effects and production style from one recording to another by example with the goal of simplifying the audio production process. We…
HEAR: Holistic Evaluation of Audio Representations
Joseph Turian, Jordie Shier, Humair Raj Khan +20
What audio embedding approach generalizes best to a wide range of downstream tasks across a variety of everyday domains without fine-tuning? The aim of the HEAR benchmark is to dev…
Steerable discovery of neural audio effects
Christian J. Steinmetz, Joshua D. Reiss
Applications of deep learning for audio effects often focus on modeling analog effects or learning to control effects to emulate a trained audio engineer. However, deep learning ap…
WaveBeat: End-to-end beat and downbeat tracking in the time domain
Christian J. Steinmetz, Joshua D. Reiss
Deep learning approaches for beat and downbeat tracking have brought advancements. However, these approaches continue to rely on hand-crafted, subsampled spectral features as input…
Direct design of biquad filter cascades with deep learning by sampling random polynomials
Joseph T. Colonel, Christian J. Steinmetz, Marcus Michelen +1
Designing infinite impulse response filters to match an arbitrary magnitude response requires specialized techniques. Methods like modified Yule-Walker are relatively efficient, bu…
Filtered Noise Shaping for Time Domain Room Impulse Response Estimation From Reverberant Speech
Christian J. Steinmetz, Vamsi Krishna Ithapu, Paul Calamia
Deep learning approaches have emerged that aim to transform an audio signal so that it sounds as if it was recorded in the same room as a reference recording, with applications bot…
Efficient neural networks for real-time modeling of analog dynamic range compression
Christian J. Steinmetz, Joshua D. Reiss
Deep learning approaches have demonstrated success in modeling analog audio effects. Nevertheless, challenges remain in modeling more complex effects that involve time-varying nonl…
Automatic multitrack mixing with a differentiable mixing console of neural audio effects
Christian J. Steinmetz, Jordi Pons, Santiago Pascual +1
Applications of deep learning to automatic multitrack mixing are largely unexplored. This is partly due to the limited available data, coupled with the fact that such data is relat…
Randomized Overdrive Neural Networks
Christian J. Steinmetz, Joshua D. Reiss
By processing audio signals in the time-domain with randomly weighted temporal convolutional networks (TCNs), we uncover a wide range of novel, yet controllable overdrive effects.…