Improving Compositional Generalization in Classification Tasks via Structure Annotations
arXiv:2106.10434
Abstract
Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. Although humans seem to have a great ability to generalize compositionally, state-of-the-art neural models struggle to do so. In this work, we study compositional generalization in classification tasks and present two main contributions. First, we study ways to convert a natural language sequence-to-sequence dataset to a classification dataset that also requires compositional generalization. Second, we show that providing structural hints (specifically, providing parse trees and entity links as attention masks for a Transformer model) helps compositional generalization.
Accepted as a short paper at ACL 2021
References in corpus (4)
- TensorFlow: A system for large-scale machine learning
- Compositional generalization in a deep seq2seq model by separating syntax and semantics
- Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures
- Memorize or generalize? Searching for a compositional RNN in a haystack