104 citations · 231 across the 19 of their papers we have counts for
24 papers
An Information Extraction Study: Take In Mind the Tokenization!
Christos Theodoropoulos, Marie-Francine Moens
Current research on the advantages and trade-offs of using characters, instead of tokenized text, as input for deep learning models, has evolved substantially. New token-free model…
Evaluation Benchmarks for Spanish Sentence Representations
Vladimir Araujo, Andrés Carvallo, Souvik Kundu +6
Due to the success of pre-trained language models, versions of languages other than English have been released in recent years. This fact implies the need for resources to evaluate…
Finding Structural Knowledge in Multimodal-BERT
Victor Milewski, Miryam de Lhoneux, Marie-Francine Moens
In this work, we investigate the knowledge learned in the embeddings of multimodal-BERT models. More specifically, we probe their capabilities of storing the grammatical structure…
Dynamic Key-value Memory Enhanced Multi-step Graph Reasoning for Knowledge-based Visual Question Answering
Mingxiao Li, Marie-Francine Moens
Knowledge-based visual question answering (VQA) is a vision-language task that requires an agent to correctly answer image-related questions using knowledge that is not presented i…
Find a Way Forward: a Language-Guided Semantic Map Navigator
Zehao Wang, Mingxiao Li, Minye Wu +2
In this paper, we introduce the map-language navigation task where an agent executes natural language instructions and moves to the target position based only on a given 3D semanti…
Discrete and continuous representations and processing in deep learning: Looking forward
Ruben Cartuyvels, Graham Spinks, Marie-Francine Moens
Discrete and continuous representations of content (e.g., of language or images) have interesting properties to be explored for the understanding of or reasoning with this content…
Analysis and Prediction of NLP Models Via Task Embeddings
Damien Sileo, Marie-Francine Moens
Task embeddings are low-dimensional representations that are trained to capture task properties. In this paper, we propose MetaEval, a collection of NLP tasks. We fit a singl…
Revisiting spatio-temporal layouts for compositional action recognition
Gorjan Radevski, Marie-Francine Moens, Tinne Tuytelaars
Recognizing human actions is fundamentally a spatio-temporal reasoning problem, and should be, at least to some extent, invariant to the appearance of the human and the objects inv…
Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations
Vladimir Araujo, Andrés Villa, Marcelo Mendoza +2
Current language models are usually trained using a self-supervised scheme, where the main focus is learning representations at the word or sentence level. However, there has been…
Imposing Relation Structure in Language-Model Embeddings Using Contrastive Learning
Christos Theodoropoulos, James Henderson, Andrei C. Coman +1
Though language model text embeddings have revolutionized NLP research, their ability to capture high-level semantic information, such as relations between entities in text, is lim…
LIIR at SemEval-2021 task 6: Detection of Persuasion Techniques In Texts and Images using CLIP features
Erfan Ghadery, Damien Sileo, Marie-Francine Moens
We describe our approach for SemEval-2021 task 6 on detection of persuasion techniques in multimodal content (memes). Our system combines pretrained multimodal models (CLIP) and ch…
A Survey on Temporal Reasoning for Temporal Information Extraction from Text (Extended Abstract)
Artuur Leeuwenberg, Marie-Francine Moens
Time is deeply woven into how people perceive, and communicate about the world. Almost unconsciously, we provide our language utterances with temporal cues, like verb tenses, and w…
Talk2Car: Taking Control of Your Self-Driving Car
Thierry Deruyttere, Simon Vandenhende, Dusan Grujicic +2
A long-term goal of artificial intelligence is to have an agent execute commands communicated through natural language. In many cases the commands are grounded in a visual environm…
Attention-based Fusion for Outfit Recommendation
Katrien Laenen, Marie-Francine Moens
This paper describes an attention-based fusion method for outfit recommendation which fuses the information in the product image and description to capture the most important, fine…
Justifying Diagnosis Decisions by Deep Neural Networks
Graham Spinks, Marie-Francine Moens
An integrated approach is proposed across visual and textual data to both determine and justify a medical diagnosis by a neural network. As deep learning techniques improve, intere…
Word-Level Loss Extensions for Neural Temporal Relation Classification
Artuur Leeuwenberg, Marie-Francine Moens
Unsupervised pre-trained word embeddings are used effectively for many tasks in natural language processing to leverage unlabeled textual data. Often these embeddings are either us…
Temporal Information Extraction by Predicting Relative Time-lines
Artuur Leeuwenberg, Marie-Francine Moens
The current leading paradigm for temporal information extraction from text consists of three phases: (1) recognition of events and temporal expressions, (2) recognition of temporal…
Generating Continuous Representations of Medical Texts
Graham Spinks, Marie-Francine Moens
We present an architecture that generates medical texts while learning an informative, continuous representation with discriminative features. During training the input to the syst…
Acquiring Common Sense Spatial Knowledge through Implicit Spatial Templates
Guillem Collell, Luc Van Gool, Marie-Francine Moens
Spatial understanding is a fundamental problem with wide-reaching real-world applications. The representation of spatial knowledge is often modeled with spatial templates, i.e., re…
Speech-Based Visual Question Answering
Ted Zhang, Dengxin Dai, Tinne Tuytelaars +2
This paper introduces speech-based visual question answering (VQA), the task of generating an answer given an image and a spoken question. Two methods are studied: an end-to-end, d…
Improving Implicit Semantic Role Labeling by Predicting Semantic Frame Arguments
Quynh Ngoc Thi Do, Steven Bethard, Marie-Francine Moens
Implicit semantic role labeling (iSRL) is the task of predicting the semantic roles of a predicate that do not appear as explicit arguments, but rather regard common sense knowledg…
Learning to Predict: A Fast Re-constructive Method to Generate Multimodal Embeddings
Guillem Collell, Teddy Zhang, Marie-Francine Moens
Integrating visual and linguistic information into a single multimodal representation is an unsolved problem with wide-reaching applications to both natural language processing and…
Deep Embedding for Spatial Role Labeling
Oswaldo Ludwig, Xiao Liu, Parisa Kordjamshidi +1
This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft C…
Bilingual Distributed Word Representations from Document-Aligned Comparable Data
Ivan Vulić, Marie-Francine Moens
We propose a new model for learning bilingual word representations from non-parallel document-aligned data. Following the recent advances in word representation learning, our model…