Log-sum-exp neural networks and posynomial models for convex and log-log-convex data

Abstract

We show in this paper that a one-layer feedforward neural network with exponential activation functions in the inner layer and logarithmic activation in the output neuron is an universal approximator of convex functions. Such a network represents a family of scaled log-sum exponential functions, here named LSET. Under a suitable exponential transformation, the class of LSET functions maps to a family of generalized posynomials GPOST, which we similarly show to be universal approximators for log-log-convex functions. A key feature of an LSET network is that, once it is trained on data, the resulting model is convex in the variables, which makes it readily amenable to efficient design based on convex optimization. Similarly, once a GPOST model is trained on data, it yields a posynomial model that can be efficiently optimized with respect to its variables by using geometric programming (GP). The proposed methodology is illustrated by two numerical examples, in which, first, models are constructed from simulation data of the two physical processes (namely, the level of vibration in a vehicle suspension system, and the peak power generated by the combustion of propane), and then optimization-based design is performed on these models.

References in corpus (6)

• A maximin characterization of the escape rate of nonexpansive mappings in metrically convex spaces
• Definable zero-sum stochastic games
• Strategy iteration is strongly polynomial for 2-player turn-based stochastic games with a constant discount factor
• A convergent hierarchy of non-linear eigenproblems to compute the joint spectral radius of nonnegative matrices
• Entropy Games and Matrix Multiplication Games
• Hourglass alternative and the finiteness conjecture for the spectral characteristics of sets of non-negative matrices

Cited by in corpus (14)

• Polyconvex anisotropic hyperelasticity with neural networks
• A Tropical Approach to Neural Networks with Piecewise Linear Activations
• Understanding and monitoring the evolution of the Covid-19 epidemic from medical emergency calls: the example of the Paris area
• Some open questions on morphological operators and representations in the deep learning era
• Tropical Polynomial Division and Neural Networks
• Accurate Computation of the Log-Sum-Exp and Softmax Functions
• Differentiating through Log-Log Convex Programs
• Parameterized Convex Universal Approximators for Decision-Making Problems
• Gradient Networks
• Advances in the training, pruning and enforcement of shape constraints of Morphological Neural Networks using Tropical Algebra
• Tropical Geometry and Piecewise-Linear Approximation of Curves and Surfaces on Weighted Lattices
• Min-Max-Plus Neural Networks
• Disciplined Geometric Programming
• Stabilising the Metzler matrices with applications to dynamical systems