105 citations · 226 across the 19 of their papers we have counts for
19 papers
A Recipe for Efficient SBIR Models: Combining Relative Triplet Loss with Batch Normalization and Knowledge Distillation
Omar Seddati, Nathan Hubens, Stéphane Dupont +1
Sketch-Based Image Retrieval (SBIR) is a crucial task in multimedia retrieval, where the goal is to retrieve a set of images that match a given sketch query. Researchers have alrea…
Transformers and CNNs both Beat Humans on SBIR
Omar Seddati, Stéphane Dupont, Saïd Mahmoudi +1
Sketch-based image retrieval (SBIR) is the task of retrieving natural images (photos) that match the semantics and the spatial configuration of hand-drawn sketch queries. The unive…
Analysis of Co-Laughter Gesture Relationship on RGB videos in Dyadic Conversation Contex
Hugo Bohy, Ahmad Hammoudeh, Antoine Maiorca +2
The development of virtual agents has enabled human-avatar interactions to become increasingly rich and varied. Moreover, an expressive virtual agent i.e. that mimics the natural e…
Deep soccer captioning with transformer: dataset, semantics-related losses, and multi-level evaluation
Ahmad Hammoudeh, Bastien Vanderplaetse, Stéphane Dupont
This work aims at generating captions for soccer videos using deep learning. In this context, this paper introduces a dataset, model, and triple-level evaluation. The dataset consi…
Multi-level Attention Fusion Network for Audio-visual Event Recognition
Mathilde Brousmiche, Jean Rouat, Stéphane Dupont
Event classification is inherently sequential and multimodal. Therefore, deep neural models need to dynamically focus on the most relevant time window and/or modality of a video. I…
Improved Soccer Action Spotting using both Audio and Video Streams
Bastien Vanderplaetse, Stéphane Dupont
In this paper, we propose a study on multi-modal (audio and video) action spotting and classification in soccer videos. Action spotting and classification are the tasks that consis…
Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognition
Jean-Benoit Delbrouck, Noé Tits, Stéphane Dupont
This paper aims to bring a new lightweight yet powerful solution for the task of Emotion Recognition and Sentiment Analysis. Our motivation is to propose two architectures based on…
AVECL-UMONS database for audio-visual event classification and localization
Mathilde Brousmiche, Stéphane Dupont, Jean Rouat
We introduce the AVECL-UMons dataset for audio-visual event classification and localization in the context of office environments. The audio-visual dataset is composed of 11 event…
A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis
Jean-Benoit Delbrouck, Noé Tits, Mathilde Brousmiche +1
Understanding expressed sentiment and emotions are two crucial factors in human multimodal language. This paper describes a Transformer-based joint-encoding (TBJE) for the task of…
Can adversarial training learn image captioning ?
Jean-Benoit Delbrouck, Bastien Vanderplaetse, Stéphane Dupont
Recently, generative adversarial networks (GAN) have gathered a lot of interest. Their efficiency in generating unseen samples of high quality, especially images, has improved over…
Modulated Self-attention Convolutional Network for VQA
Jean-Benoit Delbrouck, Antoine Maiorca, Nathan Hubens +1
As new data-sets for real-world visual reasoning and compositional question answering are emerging, it might be needed to use the visual feature extraction as a end-to-end process…
Adversarial reconstruction for Multi-modal Machine Translation
Jean-Benoit Delbrouck, Stéphane Dupont
Even with the growing interest in problems at the intersection of Computer Vision and Natural Language, grounding (i.e. identifying) the components of a structured description in a…
Object-oriented Targets for Visual Navigation using Rich Semantic Representations
Jean-Benoit Delbrouck, Stéphane Dupont
When searching for an object humans navigate through a scene using semantic information and spatial relationships. We look for an object using our knowledge of its attributes and r…
Bringing back simplicity and lightliness into neural image captioning
Jean-Benoit Delbrouck, Stéphane Dupont
Neural Image Captioning (NIC) or neural caption generation has attracted a lot of attention over the last few years. Describing an image with a natural language has been an emergin…
UMONS Submission for WMT18 Multimodal Translation Task
Jean-Benoit Delbrouck, Stéphane Dupont
This paper describes the UMONS solution for the Multimodal Machine Translation Task presented at the third conference on machine translation (WMT18). We explore a novel architectur…
Modulating and attending the source image during encoding improves Multimodal Translation
Jean-Benoit Delbrouck, Stéphane Dupont
We propose a new and fully end-to-end approach for multimodal translation where the source text encoder modulates the entire visual input processing using conditional batch normali…
An empirical study on the effectiveness of images in Multimodal Neural Machine Translation
Jean-Benoit Delbrouck, Stéphane Dupont
In state-of-the-art Neural Machine Translation (NMT), an attention mechanism is used during decoding to enhance the translation. At every step, the decoder uses this mechanism to f…
Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation
Jean-Benoit Delbrouck, Stéphane Dupont, Omar Seddati
In Multimodal Neural Machine Translation (MNMT), a neural model generates a translated sentence that describes an image, given the image itself and one source descriptions in Engli…
Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translation
Jean-Benoit Delbrouck, Stephane Dupont
In state-of-the-art Neural Machine Translation, an attention mechanism is used during decoding to enhance the translation. At every step, the decoder uses this mechanism to focus o…