Convexification of Neural Graph
arXiv:1801.02901
Abstract
Traditionally, most complex intelligence architectures are extremely non-convex, which could not be well performed by convex optimization. However, this paper decomposes complex structures into three types of nodes: operators, algorithms and functions. Iteratively, propagating from node to node along edge, we prove that "regarding the tree-structured neural graph, it is nearly convex in each variable, when the other variables are fixed." In fact, the non-convex properties stem from circles and functions, which could be transformed to be convex with our proposed \textit{\textbf{scale mechanism}}. Experimentally, we justify our theoretical analysis by two practical applications.
References in corpus (6)
- ADADELTA: An Adaptive Learning Rate Method
- LSTM-based Deep Learning Models for Non-factoid Answer Selection
- Bilateral Multi-Perspective Matching for Natural Language Sentences
- Semi-supervised Clustering for Short Text via Deep Representation Learning
- NDT: Neual Decision Tree Towards Fully Functioned Neural Graph
- Hungarian Layer: Logics Empowered Neural Architecture