121 citations · 158 across the 19 of their papers we have counts for
19 papers
ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
Aavash Chhetri, Bibek Niroula, Eduard Vazquez +4
In this paper, we address the problem of multimodal federated learning with missing modality. Existing methods utilize an additional public dataset or perform naive feature synthes…
Interactive Episodic Memory with User Feedback
Nikesh Subedi, Loris Bazzani, Ziad Al-Halah
In episodic memory with natural language queries (EM-NLQ), a user may ask a question (e.g., "Where did I place the mug?") that requires searching a long egocentric video, captured…
Benchmarking Interaction, Beyond Policy: a Reproducible Benchmark for Collaborative Instance Object Navigation
Edoardo Zorzi, Francesco Taioli, Yiming Wang +4
We propose Question-Asking Navigation (QAsk-Nav), the first reproducible benchmark for Collaborative Instance Object Navigation (CoIN) that enables an explicit, separate assessment…
Multi-Level Conditioning by Pairing Localized Text and Sketch for Fashion Image Generation
Ziyue Liu, Davide Talon, Federico Girella +5
Sketches offer designers a concise yet expressive medium for early-stage fashion ideation by specifying structure, silhouette, and spatial relationships, while textual descriptions…
Med-MMFL: A Multimodal Federated Learning Benchmark in Healthcare
Aavash Chhetri, Bibek Niroula, Pratik Shrestha +5
Federated learning (FL) enables collaborative model training across decentralized medical institutions while preserving data privacy. However, medical FL benchmarks remain scarce,…
ToFu: Visual Tokens Reduction via Fusion for Multi-modal, Multi-patch, Multi-image Task
Vittorio Pippi, Matthieu Guillaumin, Silvia Cascianelli +3
Large Multimodal Models (LMMs) are powerful tools that are capable of reasoning and understanding multimodal information beyond text and language. Despite their entrenched impact,…
UniCoRN: Unified Commented Retrieval Network with LMMs
Maximilian Jaritz, Matthieu Guillaumin, Sabine Sternig +1
Multimodal retrieval methods have limitations in handling complex, compositional queries that require reasoning about the visual content of both the query and the retrieved entitie…
Learning Visual Hierarchies in Hyperbolic Space for Image Retrieval
Ziwei Wang, Sameera Ramasinghe, Chenchen Xu +3
Structuring latent representations in a hierarchical manner enables models to learn patterns at multiple levels of abstraction. However, most prevalent image understanding models f…
LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts
Anh-Quan Cao, Maximilian Jaritz, Matthieu Guillaumin +2
Large-scale vision-language pre-trained (VLP) models (e.g., CLIP) are renowned for their versatility, as they can be applied to diverse applications in a zero-shot setup. However,…
ViewFusion: Towards Multi-View Consistency via Interpolated Denoising
Xianghui Yang, Yan Zuo, Sameera Ramasinghe +3
Novel-view synthesis through diffusion models has demonstrated remarkable potential for generating diverse and high-quality images. Yet, the independent process of image generation…
iEdit: Localised Text-guided Image Editing with Weak Supervision
Rumeysa Bodur, Erhan Gundogdu, Binod Bhattarai +3
Diffusion models (DMs) can generate realistic images with text guidance using large-scale datasets. However, they demonstrate limited controllability in the output space of the gen…
Contrastive Language-Action Pre-training for Temporal Localization
Mengmeng Xu, Erhan Gundogdu, Maksim Lapin +3
Long-form video understanding requires designing approaches that are able to temporally localize activities or language. End-to-end training for such tasks is limited by the comput…
Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning
Amaia Salvador, Erhan Gundogdu, Loris Bazzani +1
Cross-modal recipe retrieval has recently gained substantial attention due to the importance of food in people's lives, as well as the availability of vast amounts of digital cooki…
Image Captioning as Neural Machine Translation Task in SOCKEYE
Loris Bazzani, Tobias Domhan, Felix Hieber
Image captioning is an interdisciplinary research problem that stands between computer vision and natural language processing. The task is to generate a textual description of the…
Kernel Methods on Approximate Infinite-Dimensional Covariance Operators for Image Classification
Hà Quang Minh, Marco San Biagio, Loris Bazzani +1
This paper presents a novel framework for visual object recognition using infinite-dimensional covariance operators of input features in the paradigm of kernel methods on infinite-…
Recurrent Mixture Density Network for Spatiotemporal Visual Attention
Loris Bazzani, Hugo Larochelle, Lorenzo Torresani
In many computer vision tasks, the relevant information to solve the problem at hand is mixed to irrelevant, distracting information. This has motivated researchers to design atten…
Self-taught Object Localization with Deep Networks
Loris Bazzani, Alessandro Bergamo, Dragomir Anguelov +1
This paper introduces self-taught object localization, a novel approach that leverages deep convolutional networks trained for whole-image recognition to localize objects in images…
A Unifying Framework in Vector-valued Reproducing Kernel Hilbert Spaces for Manifold Regularization and Co-Regularized Multi-view Learning
Ha Quang Minh, Loris Bazzani, Vittorio Murino
This paper presents a general vector-valued reproducing kernel Hilbert spaces (RKHS) framework for the problem of learning an unknown functional dependency between a structured inp…
Learning where to Attend with Deep Architectures for Image Tracking
Misha Denil, Loris Bazzani, Hugo Larochelle +1
We discuss an attentional model for simultaneous object tracking and recognition that is driven by gaze data. Motivated by theories of perception, the model consists of two interac…