Neuro-SERKET: Development of Integrative Cognitive System through the Composition of Deep Probabilistic Generative Models

Abstract

This paper describes a framework for the development of an integrative cognitive system based on probabilistic generative models (PGMs) called Neuro-SERKET. Neuro-SERKET is an extension of SERKET, which can compose elemental PGMs developed in a distributed manner and provide a scheme that allows the composed PGMs to learn throughout the system in an unsupervised way. In addition to the head-to-tail connection supported by SERKET, Neuro-SERKET supports tail-to-tail and head-to-head connections, as well as neural network-based modules, i.e., deep generative models. As an example of a Neuro-SERKET application, an integrative model was developed by composing a variational autoencoder (VAE), a Gaussian mixture model (GMM), latent Dirichlet allocation (LDA), and automatic speech recognition (ASR). The model is called VAE+GMM+LDA+ASR. The performance of VAE+GMM+LDA+ASR and the validity of Neuro-SERKET were demonstrated through a multimodal categorization task using image data and a speech signal of numerical digits.

arXiv commentNew Gener. Comput. (2020)

References in corpus (5)

• Symbol Emergence in Robotics: A Survey
• Symbol Emergence in Cognitive Developmental Systems: a Survey
• Online Spatial Concept and Lexical Acquisition with Simultaneous Localization and Mapping
• Nonparametric Bayesian Double Articulation Analyzer for Direct Language Acquisition from Continuous Speech Signals
• Unsupervised Phoneme and Word Discovery from Multiple Speakers using Double Articulation Analyzer and Neural Network with Parametric Bias

Cited by in corpus (5)

• A survey of multimodal deep generative models
• Hierarchical Path-planning from Speech Instructions with Spatial Concept-based Topometric Semantic Mapping
• On Parallelism in Music and Language: A Perspective from Symbol Emergence Systems based on Probabilistic Generative Models
• Pixyz: a Python library for developing deep generative models
• Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues