Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing

Abstract

Predicting linearized Abstract Meaning Representation (AMR) graphs using pre-trained sequence-to-sequence Transformer models has recently led to large improvements on AMR parsing benchmarks. These parsers are simple and avoid explicit modeling of structure but lack desirable properties such as graph well-formedness guarantees or built-in graph-sentence alignments. In this work we explore the integration of general pre-trained sequence-to-sequence language models and a structure-aware transition-based approach. We depart from a pointer-based transition system and propose a simplified transition set, designed to better exploit pre-trained language models for structured fine-tuning. We also explore modeling the parser state within the pre-trained encoder-decoder architecture and different vocabulary strategies for the same purpose. We provide a detailed comparison with recent progress in AMR parsing and show that the proposed parser retains the desirable properties of previous transition-based approaches, while being simpler and reaching the new parsing state of the art for AMR 2.0, without the need for graph re-categorization.

arXiv commentEMNLP 2021 main conference

References in corpus (8)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• fairseq: A Fast, Extensible Toolkit for Sequence Modeling
• Neural Semantic Parsing by Character-based Translation: Experiments with Abstract Meaning Representations
• Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation
• AMR Parsing as Sequence-to-Graph Transduction
• A Hierarchy of Graph Neural Networks Based on Learnable Local Features
• Improving AMR Parsing with Sequence-to-Sequence Pre-training
• Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing