TimbreCLIP: Connecting Timbre to Text and Images

Abstract

We present work in progress on TimbreCLIP, an audio-text cross modal embedding trained on single instrument notes. We evaluate the models with a cross-modal retrieval task on synth patches. Finally, we demonstrate the application of TimbreCLIP on two tasks: text-driven audio equalization and timbre to image generation.

Submitted to AAAI workshop on creative AI across modalities