Can LSTM Learn to Capture Agreement? The Case of Basque

Abstract

Sequential neural networks models are powerful tools in a variety of Natural Language Processing (NLP) tasks. The sequential nature of these models raises the questions: to what extent can these models implicitly learn hierarchical structures typical to human language, and what kind of grammatical phenomena can they acquire? We focus on the task of agreement prediction in Basque, as a case study for a task that requires implicit understanding of sentence structure and the acquisition of a complex but consistent morphological system. Analyzing experimental results from two syntactic prediction tasks -- verb number prediction and suffix recovery -- we find that sequential models perform worse on agreement prediction in Basque than one might expect on the basis of a previous agreement prediction work in English. Tentative findings based on diagnostic classifiers suggest the network makes use of local heuristics as a proxy for the hierarchical structure of the sentence. We propose the Basque agreement prediction task as challenging benchmark for models that attempt to learn regularities in human language.

Accepted to "Analyzing and interpreting neural networks for NLP" workshop at EMNLP 2018

References in corpus (5)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
• Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
• On the State of the Art of Evaluation in Neural Language Models

Cited by in corpus (3)

• A Systematic Analysis of Morphological Content in BERT Models for Multiple Languages
• Understanding Cross-Lingual Syntactic Transfer in Multilingual Recurrent Neural Networks
• Recurrent babbling: evaluating the acquisition of grammar from limited input data