An Adaptive Learning Method of Deep Belief Network by Layer Generation Algorithm

Abstract

Deep Belief Network (DBN) has a deep architecture that represents multiple features of input patterns hierarchically with the pre-trained Restricted Boltzmann Machines (RBM). A traditional RBM or DBN model cannot change its network structure during the learning phase. Our proposed adaptive learning method can discover the optimal number of hidden neurons and weights and/or layers according to the input space. The model is an important method to take account of the computational cost and the model stability. The regularities to hold the sparse structure of network is considerable problem, since the extraction of explicit knowledge from the trained network should be required. In our previous research, we have developed the hybrid method of adaptive structural learning method of RBM and Learning Forgetting method to the trained RBM. In this paper, we propose the adaptive learning method of DBN that can determine the optimal number of layers during the learning. We evaluated our proposed model on some benchmark data sets.

4 pages, 2 figures, Proc. of 2016 IEEE Region 10 Conference (TENCON2016)

References in corpus (2)

• An Adaptive Learning Method of Restricted Boltzmann Machine by Neuron Generation and Annihilation Algorithm
• Fine Tuning Method by using Knowledge Acquisition from Deep Belief Network

Cited by in corpus (9)

• Fine Tuning Method by using Knowledge Acquisition from Deep Belief Network
• Adaptive Learning Method of Recurrent Temporal Deep Belief Network to Analyze Time Series Data
• An Adaptive Structural Learning of Deep Belief Network for Image-based Crack Detection in Concrete Structures Using SDNET2018
• Knowledge Extracted from Recurrent Deep Belief Network for Real Time Deterministic Control
• Automatic Extraction of Road Networks by using Teacher-Student Adaptive Structural Deep Belief Network and Its Application to Landslide Disaster
• An Embedded System for Image-based Crack Detection by using Fine-Tuning model of Adaptive Structural Learning of Deep Belief Network
• Shortening Time Required for Adaptive Structural Learning Method of Deep Belief Network with Multi-Modal Data Arrangement
• Automatic Extraction of Road Networks from Satellite Images by using Adaptive Structural Deep Belief Network
• A Distillation Learning Model of Adaptive Structural Deep Belief Network for AffectNet: Facial Expression Image Database