SpecSinGAN: Sound Effect Variation Synthesis Using Single-Image GANs
arXiv:2110.07311
Abstract
Single-image generative adversarial networks learn from the internal distribution of a single training example to generate variations of it, removing the need of a large dataset. In this paper we introduce SpecSinGAN, an unconditional generative architecture that takes a single one-shot sound effect (e.g., a footstep; a character jump) and produces novel variations of it, as if they were different takes from the same recording session. We explore the use of multi-channel spectrograms to train the model on the various layers that comprise a single sound effect. A listening study comparing our model to real recordings and to digital signal processing procedural audio models in terms of sound plausibility and variation revealed that SpecSinGAN is more plausible and varied than the procedural audio models considered, when using multi-channel spectrograms. Sound examples can be found at the project website: https://www.adrianbarahonarios.com/specsingan/
References in corpus (6)
- GANSynth: Adversarial Neural Audio Synthesis
- DiffWave: A Versatile Diffusion Model for Audio Synthesis
- DDSP: Differentiable Digital Signal Processing
- DrumGAN: Synthesis of Drum Sounds With Timbral Feature Conditioning Using Generative Adversarial Networks
- Catch-A-Waveform: Learning to Generate Audio from a Single Short Example
- Real-time Timbre Transfer and Sound Synthesis using DDSP