Iterative Implicit Gradients for Nonconvex Optimization with Variational Inequality Constraints

Abstract

We propose an optimization proxy in terms of iterative implicit gradient methods for solving constrained optimization problems with nonconvex loss functions. This framework can be applied to a broad range of machine learning settings, including meta-learning, hyperparameter optimization, large-scale complicated constrained optimization, and reinforcement learning. The proposed algorithm builds upon the iterative differentiation (ITD) approach. We extend existing convergence and rate analyses from the bilevel optimization literature to a constrained bilevel setting, motivated by learning under explicit constraints. Since solving bilevel problems using first-order methods requires evaluating the gradient of the inner-level optimal solution with respect to the outer variable (the implicit gradient), we develop an efficient computation strategy suitable for large-scale structures. Furthermore, we establish error bounds relative to the true gradients and provide non-asymptotic convergence rate guarantees.