Fuzzy Logic Interpretation of Quadratic Networks

Abstract

Over past several years, deep learning has achieved huge successes in various applications. However, such a data-driven approach is often criticized for lack of interpretability. Recently, we proposed artificial quadratic neural networks consisting of second-order neurons in potentially many layers. In each second-order neuron, a quadratic function is used in the place of the inner product in a traditional neuron, and then undergoes a nonlinear activation. With a single second-order neuron, any fuzzy logic operation, such as XOR, can be implemented. In this sense, any deep network constructed with quadratic neurons can be interpreted as a deep fuzzy logic system. Since traditional neural networks and second-order counterparts can represent each other and fuzzy logic operations are naturally implemented in second-order neural networks, it is plausible to explain how a deep neural network works with a second-order network as the system model. In this paper, we generalize and categorize fuzzy logic operations implementable with individual second-order neurons, and then perform statistical/information theoretic analyses of exemplary quadratic neural networks.

10 pages and 9 figures

References in corpus (15)

• Neural Architecture Search with Reinforcement Learning
• Understanding Neural Networks Through Deep Visualization
• Understanding Black-box Predictions via Influence Functions
• Designing Neural Network Architectures using Reinforcement Learning
• Structure-sensitive Multi-scale Deep Neural Network for Low-Dose CT Denoising
• Qualitatively characterizing neural network optimization problems
• The loss surface of deep and wide neural networks
• An exact mapping between the Variational Renormalization Group and Deep Learning
• Interpreting Blackbox Models via Model Extraction
• An unexpected unity among methods for interpreting model predictions
• Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
• On the Computational Efficiency of Training Neural Networks
• Diverse Neural Network Learns True Target Functions
• Global optimality conditions for deep neural networks
• Geometric Understanding of Deep Learning