6 papers
Separating Retention from Extraction in the Evaluation of End-to-end Relation Extraction
Bruno Taillé, Vincent Guigue, Geoffrey Scoutheeten +1
State-of-the-art NLP models can adopt shallow heuristics that limit their generalization capability (McCoy et al., 2019). Such heuristics include lexical overlap with the training…
Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation
Clément Rebuffel, Thomas Scialom, Laure Soulier +5
QuestEval is a reference-less metric used in text-to-text tasks, that compares the generated summaries directly to the source text, by automatically asking and answering questions.…
Controlling Hallucinations at Word Level in Data-to-Text Generation
Clément Rebuffel, Marco Roberti, Laure Soulier +3
Data-to-Text Generation (DTG) is a subfield of Natural Language Generation aiming at transcribing structured data in natural language descriptions. The field has been recently boos…
PARENTing via Model-Agnostic Reinforcement Learning to Correct Pathological Behaviors in Data-to-Text Generation
Clément Rebuffel, Laure Soulier, Geoffrey Scoutheeten +1
In language generation models conditioned by structured data, the classical training via maximum likelihood almost always leads models to pick up on dataset divergence (i.e., hallu…
Let's Stop Incorrect Comparisons in End-to-end Relation Extraction!
Bruno Taillé, Vincent Guigue, Geoffrey Scoutheeten +1
Despite efforts to distinguish three different evaluation setups (Bekoulis et al., 2018), numerous end-to-end Relation Extraction (RE) articles present unreliable performance compa…
A Hierarchical Model for Data-to-Text Generation
Clément Rebuffel, Laure Soulier, Geoffrey Scoutheeten +1
Transcribing structured data into natural language descriptions has emerged as a challenging task, referred to as "data-to-text". These structures generally regroup multiple elemen…