Imposing Hard Constraints on Deep Networks: Promises and Limitations
arXiv:1706.02025
Abstract
Imposing constraints on the output of a Deep Neural Net is one way to improve the quality of its predictions while loosening the requirements for labeled training data. Such constraints are usually imposed as soft constraints by adding new terms to the loss function that is minimized during training. An alternative is to impose them as hard constraints, which has a number of theoretical benefits but has not been explored so far due to the perceived intractability of the problem. In this paper, we show that imposing hard constraints can in fact be done in a computationally feasible way and delivers reasonable results. However, the theoretical benefits do not materialize and the resulting technique is no better than existing ones relying on soft constraints. We analyze the reasons for this and hope to spur other researchers into proposing better solutions.
References in corpus (1)
Cited by in corpus (9)
- Physics informed deep learning for computational elastodynamics without labeled data
- Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey
- Deep learning calibration of option pricing models: some pitfalls and solutions
- Convergence of adaptive algorithms for weakly convex constrained optimization
- Inexact Sequential Quadratic Optimization for Minimizing a Stochastic Objective Function Subject to Deterministic Nonlinear Equality Constraints
- Deep Local Volatility
- Distortion-controlled Training for End-to-end Reverberant Speech Separation with Auxiliary Autoencoding Loss
- One Size Fits All: Can We Train One Denoiser for All Noise Levels?
- Fast Jacobian-Vector Product for Deep Networks