4 papers · 1 filter
Controllable Audio-Visual Viewpoint Generation from 360° Spatial Information
Christian Marinoni, Riccardo Fosco Gramaccioni, Eleonora Grassucci +1
The generation of sounding videos has seen significant advancements with the advent of diffusion models. However, existing methods often lack the fine-grained control needed to gen…
FoleyGRAM: Video-to-Audio Generation with GRAM-Aligned Multimodal Encoders
Riccardo Fosco Gramaccioni, Christian Marinoni, Eleonora Grassucci +3
In this work, we present FoleyGRAM, a novel approach to video-to-audio generation that emphasizes semantic conditioning through the use of aligned multimodal encoders. Building on…
A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similarity
Giordano Cicchetti, Eleonora Grassucci, Danilo Comminiello
Multimodal learning plays a pivotal role in advancing artificial intelligence systems by incorporating information from multiple modalities to build a more comprehensive representa…
Semantic Compression via Multimodal Representation Learning
Eleonora Grassucci, Giordano Cicchetti, Aurelio Uncini +1
Multimodal representation learning produces high-dimensional embeddings that align diverse modalities in a shared latent space. While this enables strong generalization, it also in…