3 citations · 4 across the 5 of their papers we have counts for
12 papers · 1 filter
Better Smatch = Better Parser? AMR evaluation is not so simple anymore
Juri Opitz, Anette Frank
Recently, astonishing advances have been observed in AMR parsing, as measured by the structural Smatch metric. In fact, today's systems achieve performance levels that seem to surp…
A Dynamic, Interpreted CheckList for Meaning-oriented NLG Metric Evaluation -- through the Lens of Semantic Similarity Rating
Laura Zeidler, Juri Opitz, Anette Frank
Evaluating the quality of generated text is difficult, since traditional NLG evaluation metrics, focusing more on surface form than meaning, often fail to assign appropriate scores…
Weisfeiler-Leman in the BAMBOO: Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity
Juri Opitz, Angel Daza, Anette Frank
Several metrics have been proposed for assessing the similarity of (abstract) meaning representations (AMRs), but little is known about how they relate to human similarity ratings.…
Translate, then Parse! A strong baseline for Cross-Lingual AMR Parsing
Sarah Uhrig, Yoalli Rezepka Garcia, Juri Opitz +1
In cross-lingual Abstract Meaning Representation (AMR) parsing, researchers develop models that project sentences from various languages onto their AMRs to capture their essential…
Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR
Juri Opitz, Anette Frank
Systems that generate natural language text from abstract meaning representations such as AMR are typically evaluated using automatic surface matching metrics that compare the gene…
AMR Quality Rating with a Lightweight CNN
Juri Opitz
Structured semantic sentence representations such as Abstract Meaning Representations (AMRs) are potentially useful in various NLP tasks. However, the quality of automatic parses c…