Stochastic Configuration Networks: Fundamentals and Algorithms

Abstract

This paper contributes to a development of randomized methods for neural networks. The proposed learner model is generated incrementally by stochastic configuration (SC) algorithms, termed as Stochastic Configuration Networks (SCNs). In contrast to the existing randomised learning algorithms for single layer feed-forward neural networks (SLFNNs), we randomly assign the input weights and biases of the hidden nodes in the light of a supervisory mechanism, and the output weights are analytically evaluated in either constructive or selective manner. As fundamentals of SCN-based data modelling techniques, we establish some theoretical results on the universal approximation property. Three versions of SC algorithms are presented for regression problems (applicable for classification problems as well) in this work. Simulation results concerning both function approximation and real world data regression indicate some remarkable merits of our proposed SCNs in terms of less human intervention on the network size setting, the scope adaptation of random parameters, fast learning and sound generalization.

Manuscript submitted to IEEE TCYB on Feb. 12, 2017; 18 pages, 4 figures and 28 references

References in corpus (1)

• Stochastic Configuration Networks Ensemble for Large-Scale Data Analytics

Cited by in corpus (26)

• Theory of neuromorphic computing by waves: machine learning by rogue waves, dispersive shocks, and solitons
• Stochastic Configuration Networks Ensemble for Large-Scale Data Analytics
• Blessing of dimensionality: mathematical foundations of the statistical physics of data
• A New Learning Paradigm for Random Vector Functional-Link Network: RVFL+
• Correction of AI systems by linear discriminants: Probabilistic foundations
• Parsimonious Random Vector Functional Link Network for Data Streams
• Density Encoding Enables Resource-Efficient Randomly Connected Neural Networks
• A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machines
• An attentive neural architecture for joint segmentation and parsing and its application to real estate ads
• Rolling the dice for better deep learning performance: A study of randomness techniques in deep neural networks
• Feature extraction and classification algorithm, which one is more essential? An experimental study on a specific task of vibration signal diagnosis
• General stochastic separation theorems with optimal bounds
• Cluster-based Input Weight Initialization for Echo State Networks
• Fast Construction of Correcting Ensembles for Legacy Artificial Intelligence Systems: Algorithms and a Case Study
• Error-feedback stochastic modeling strategy for time series forecasting with convolutional neural networks
• Blessing of dimensionality at the edge
• Improving Randomized Learning of Feedforward Neural Networks by Appropriate Generation of Random Parameters
• How Powerful are Shallow Neural Networks with Bandlimited Random Weights?
• Deep Stochastic Configuration Networks with Universal Approximation Property
• A Method of Generating Random Weights and Biases in Feedforward Neural Networks with Random Hidden Nodes
• Multi-Kernel Correntropy for Robust Learning
• Deep Stacked Stochastic Configuration Networks for Lifelong Learning of Non-Stationary Data Streams
• Robust Stochastic Configuration Networks with Kernel Density Estimation
• Are Direct Links Necessary in RVFL NNs for Regression?
• Effective Non-Random Extreme Learning Machine
• Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions