Compositionality Through Language Transmission, using Artificial Neural Networks

Abstract

We propose an architecture and process for using the Iterated Learning Model ("ILM") for artificial neural networks. We show that ILM does not lead to the same clear compositionality as observed using DCGs, but does lead to a modest improvement in compositionality, as measured by holdout accuracy and topologic similarity. We show that ILM can lead to an anti-correlation between holdout accuracy and topologic rho. We demonstrate that ILM can increase compositionality when using non-symbolic high-dimensional images as input.

arXiv commentUpdated figure layout, fixed typos, modified title

References in corpus (7)

• Distilling the Knowledge in a Neural Network
• Do Deep Nets Really Need to be Deep?
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Emergence of Grounded Compositional Language in Multi-Agent Populations
• Revisiting Self-Training for Neural Sequence Generation
• Emergence of Compositional Language with Deep Generational Transmission
• Compositional Languages Emerge in a Neural Iterated Learning Model