61 citations · 174 across the 62 of their papers we have counts for
17 papers · 1 filter
BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks
Juan Rodriguez, Xiangru Jian, Siba Smarak Panigrahi +40
Multimodal AI has the potential to significantly enhance document-understanding tasks, such as processing receipts, understanding workflows, extracting data from documents, and sum…
Visual Modality Prompt for Adapting Vision-Language Object Detectors
Heitor R. Medeiros, Atif Belal, Srikanth Muralidharan +2
The zero-shot performance of object detectors degrades when tested on different modalities, such as infrared and depth. While recent work has explored image translation techniques…
IntentGPT: Few-shot Intent Discovery with Large Language Models
Juan A. Rodriguez, Nicholas Botzer, David Vazquez +3
In today's digitally driven world, dialogue systems play a pivotal role in enhancing user interactions, from customer service to virtual assistants. In these dialogues, it is impor…
Semantic Anchor Transport: Robust Test-Time Adaptation for Vision-Language Models
Shambhavi Mishra, Julio Silva-Rodriguez, Ismail Ben Ayed +2
Large pre-trained vision-language models (VLMs), such as CLIP, have shown unprecedented zero-shot performance across a wide range of tasks. Nevertheless, these models may be unreli…
Spatial Action Unit Cues for Interpretable Deep Facial Expression Recognition
Soufiane Belharbi, Marco Pedersoli, Alessandro Lameiras Koerich +2
Although state-of-the-art classifiers for facial expression recognition (FER) can achieve a high level of accuracy, they lack interpretability, an important feature for end-users.…
Source-Free Domain Adaptation for YOLO Object Detection
Simon Varailhon, Masih Aminbeidokhti, Marco Pedersoli +1
Source-free domain adaptation (SFDA) is a challenging problem in object detection, where a pre-trained source model is adapted to a new target domain without using any source domai…
Multi Teacher Privileged Knowledge Distillation for Multimodal Expression Recognition
Muhammad Haseeb Aslam, Marco Pedersoli, Alessandro Lameiras Koerich +1
Human emotion is a complex phenomenon conveyed and perceived through facial expressions, vocal tones, body language, and physiological signals. Multimodal emotion recognition syste…
Leveraging Transformers for Weakly Supervised Object Localization in Unconstrained Videos
Shakeeb Murtaza, Marco Pedersoli, Aydin Sarraf +1
Weakly-Supervised Video Object Localization (WSVOL) involves localizing an object in videos using only video-level labels, also referred to as tags. State-of-the-art WSVOL methods…
Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild
Nicolas Richet, Soufiane Belharbi, Haseeb Aslam +8
Systems for multimodal emotion recognition (ER) are commonly trained to extract features from different modalities (e.g., visual, audio, and textual) that are combined to predict i…
Joint Multimodal Transformer for Emotion Recognition in the Wild
Paul Waligora, Haseeb Aslam, Osama Zeeshan +5
Multimodal emotion recognition (MMER) systems typically outperform unimodal systems by leveraging the inter- and intra-modal relationships between, e.g., visual, textual, physiolog…
MiPa: Mixed Patch Infrared-Visible Modality Agnostic Object Detection
Heitor R. Medeiros, David Latortue, Eric Granger +1
In real-world scenarios, using multiple modalities like visible (RGB) and infrared (IR) can greatly improve the performance of a predictive task such as object detection (OD). Mult…
A Realistic Protocol for Evaluation of Weakly Supervised Object Localization
Shakeeb Murtaza, Soufiane Belharbi, Marco Pedersoli +1
Weakly Supervised Object Localization (WSOL) allows training deep learning models for classification and localization (LOC) using only global class-level labels. The absence of bou…
Modality Translation for Object Detection Adaptation Without Forgetting Prior Knowledge
Heitor Rapela Medeiros, Masih Aminbeidokhti, Fidel Guerrero Pena +3
A common practice in deep learning involves training large neural networks on massive datasets to achieve high accuracy across various domains and tasks. While this approach works…
Do not trust what you trust: Miscalibration in Semi-supervised Learning
Shambhavi Mishra, Balamurali Murugesan, Ismail Ben Ayed +2
State-of-the-art semi-supervised learning (SSL) approaches rely on highly confident predictions to serve as pseudo-labels that guide the training on unlabeled samples. An inherent…
Attention-based Class-Conditioned Alignment for Multi-Source Domain Adaptation of Object Detectors
Atif Belal, Akhil Meethal, Francisco Perdigon Romero +2
Domain adaptation methods for object detection (OD) strive to mitigate the impact of distribution shifts by promoting feature alignment across source and target domains. Multi-sour…
Guided Interpretable Facial Expression Recognition via Spatial Action Unit Cues
Soufiane Belharbi, Marco Pedersoli, Alessandro Lameiras Koerich +2
Although state-of-the-art classifiers for facial expression recognition (FER) can achieve a high level of accuracy, they lack interpretability, an important feature for end-users.…
Distilling Privileged Multimodal Information for Expression Recognition using Optimal Transport
Muhammad Haseeb Aslam, Muhammad Osama Zeeshan, Soufiane Belharbi +4
Deep learning models for multimodal expression recognition have reached remarkable performance in controlled laboratory environments because of their ability to learn complementary…