9 papers
GRAMformer: Any-Order Modality Interactions via Volumetric Multimodal Cross-Attention
Giordano Cicchetti, Eleonora Grassucci, Danilo Comminiello
Transformer-based multimodal models rely on attention mechanisms to integrate information across heterogeneous modalities. Despite their success, existing multimodal attention form…
Training-Free Multi-User Generative Semantic Communications via Null-Space Diffusion Sampling
Eleonora Grassucci, Jinho Choi, Jihong Park +3
In recent years, novel communication strategies have emerged to face the challenges that the increased number of connected devices and the higher quality of transmitted information…
Closing the gap in multimodal medical representation alignment
Eleonora Grassucci, Giordano Cicchetti, Danilo Comminiello
In multimodal learning, CLIP has emerged as the de-facto approach for mapping different modalities into a shared latent space by bringing semantically similar representations close…
Closing the Modality Gap Aligns Group-Wise Semantics
Eleonora Grassucci, Giordano Cicchetti, Emanuele Frasca +2
In multimodal learning, CLIP has been recognized as the \textit{de facto} method for learning a shared latent space across multiple modalities, placing similar representations clos…
FoleyGRAM: Video-to-Audio Generation with GRAM-Aligned Multimodal Encoders
Riccardo Fosco Gramaccioni, Christian Marinoni, Eleonora Grassucci +3
In this work, we present FoleyGRAM, a novel approach to video-to-audio generation that emphasizes semantic conditioning through the use of aligned multimodal encoders. Building on…
A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similarity
Giordano Cicchetti, Eleonora Grassucci, Danilo Comminiello
Multimodal learning plays a pivotal role in advancing artificial intelligence systems by incorporating information from multiple modalities to build a more comprehensive representa…