Neural Morphing: Sequence-Optimized Token-Level Morphing in Neural Audio Codecs
arXiv:2607.12725
The paper introduces Neural Morphing, a training‑free technique that edits audio by selecting and replacing residual‑vector‑quantized token grains from a pretrained neural audio codec using a grouped transfer policy and a continuity‑constrained beam‑search matcher, and implements it as a real‑time VST3/AU plugin.
Abstract
Neural audio codecs were originally developed for high-fidelity compression; however, their latent token representations and expressive decoders also constitute a powerful substrate for controllable audio transformation. This work introduces Neural Morphing, a training-free token-domain audio effect that selects residual-vector-quantized (RVQ) token grains from a user palette and decodes the edited stream through a pretrained codec. The method combines an RVQ-group transfer policy that separates coarse, middle, and fine codebook groups with a continuity-constrained sequence matcher that replaces independent greedy selection with bounded beam search. The intended output is a controlled hybrid: the source preserves rhythmic organization while the palette contributes timbral color and residual detail. We focus on the implementation and realtime behavior of a deployable VST3/AU system, including chunked rendering, palette-size scaling, and backend health checks.
In proceedings of the 29th International Conference on Digital Audio Effects (DAFx) 2026