52 citations · 104 across the 23 of their papers we have counts for
25 papers
On the Depth Transfer of Neural Optimizers via L-Smoothness
Maximilian Wong, Chenyu Xu, Jiajin Li +1
Scaling deep learning models efficiently requires predictable performance transfer across compute regimes. Establishing reliable hyperparameter transfer allows optimal training dyn…
Last-Iterate Convergence of Single-Loop Stochastic Methods for Constrained Convex-Concave Minimax Problems
Taoli Zheng, Jiajin Li, Anthony Man-Cho So
In this paper, we study last-iterate convergence of stochastic first-order methods for constrained smooth convex--concave minimax optimization under the standard bounded-variance s…
Nonconvex Composite Functional Constraints via First-Order Augmented Lagrangian Methods under Local Regularity
Linglingzhi Zhu, Jiajin Li
We study nonasymptotic convergence of primal-dual methods for a class of nonconvex constrained optimization problems under local multiplier regularity. Both the objective and funct…
Doubly Smoothed Optimistic Gradients: A Universal Approach for Smooth Minimax Problems
Taoli Zheng, Anthony Man-Cho So, Jiajin Li
Smooth minimax optimization problems play a central role in a wide range of applications, including machine learning, game theory, and operations research. However, existing algori…
Set Smoothness Unlocks Clarke Hyper-stationarity in Bilevel Optimization
He Chen, Jiajin Li, Anthony Man-Cho So
Solving bilevel optimization (BLO) problems to global optimality is generally intractable. A common surrogate is to compute a hyper-stationary point -- a stationary point of the hy…
Spurious Stationarity and Hardness Results for Bregman Proximal-Type Algorithms
He Chen, Jiajin Li, Anthony Man-Cho So
Bregman proximal-type algorithms (BPs), such as mirror descent, have become popular tools in machine learning and data science for exploiting problem structures through non-Euclide…
Unifying Distributionally Robust Optimization via Optimal Transport Theory
Jose Blanchet, Daniel Kuhn, Jiajin Li +1
In recent years, two prominent paradigms have shaped distributionally robust optimization (DRO), modeling distributional ambiguity through -divergences and Wasserstein distances…
A Convergent Single-Loop Algorithm for Relaxation of Gromov-Wasserstein in Graph Data
Jiajin Li, Jianheng Tang, Lemin Kong +4
In this work, we present the Bregman Alternating Projected Gradient (BAPG) method, a single-loop algorithm that offers an approximate solution to the Gromov-Wasserstein (GW) distan…
Outlier-Robust Gromov-Wasserstein for Graph Data
Lemin Kong, Jiajin Li, Jianheng Tang +1
Gromov-Wasserstein (GW) distance is a powerful tool for comparing and aligning probability distributions supported on different metric spaces. Recently, GW has become the main mode…
Robust Attributed Graph Alignment via Joint Structure Learning and Optimal Transport
Jianheng Tang, Weiqi Zhang, Jiajin Li +3
Graph alignment, which aims at identifying corresponding entities across multiple networks, has been widely applied in various domains. As the graphs to be aligned are usually cons…
Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimization
Taoli Zheng, Linglingzhi Zhu, Anthony Man-Cho So +2
Nonconvex-nonconcave minimax optimization has received intense attention over the last decade due to its broad applications in machine learning. Most existing algorithms rely on on…
Synthetic Principal Component Design: Fast Covariate Balancing with Synthetic Controls
Yiping Lu, Jiajin Li, Lexing Ying +1
The optimal design of experiments typically involves solving an NP-hard combinatorial optimization problem. In this paper, we aim to develop a globally convergent and practically e…
Tikhonov Regularization is Optimal Transport Robust under Martingale Constraints
Jiajin Li, Sirui Lin, Jose Blanchet +1
Distributionally robust optimization has been shown to offer a principled way to regularize learning models. In this paper, we find that Tikhonov regularization is distributionally…
Nonsmooth Nonconvex-Nonconcave Minimax Optimization: Primal-Dual Balancing and Iteration Complexity Analysis
Jiajin Li, Linglingzhi Zhu, Anthony Man-Cho So
Nonconvex-nonconcave minimax optimization has gained widespread interest over the last decade. However, most existing works focus on variants of gradient descent-ascent (GDA) algor…
Rethinking Graph Neural Networks for Anomaly Detection
Jianheng Tang, Jiajin Li, Ziqi Gao +1
Graph Neural Networks (GNNs) are widely applied for graph anomaly detection. As one of the key components for GNN design is to select a tailored spectral filter, we take the first…
Fast and Provably Convergent Algorithms for Gromov-Wasserstein in Graph Data
Jiajin Li, Jianheng Tang, Lemin Kong +4
In this paper, we study the design and analysis of a class of efficient algorithms for computing the Gromov-Wasserstein (GW) distance tailored to large-scale graph learning tasks.…
Learning Proximal Operators to Discover Multiple Optima
Lingxiao Li, Noam Aigerman, Vladimir G. Kim +4
Finding multiple solutions of non-convex optimization problems is a ubiquitous yet challenging task. Most past algorithms either apply single-solution optimization methods from mul…
Deconvolutional Networks on Graph Data
Jia Li, Jiajin Li, Yang Liu +3
In this paper, we consider an inverse problem in graph learning domain -- ``given the graph representations smoothed by Graph Convolutional Network (GCN), how can we reconstruct th…
A Splitting Scheme for Flip-Free Distortion Energies
Oded Stein, Jiajin Li, Justin Solomon
We introduce a robust optimization method for flip-free distortion energies used, for example, in parametrization, deformation, and volume correspondence. This method can minimize…
Dirichlet Graph Variational Autoencoder
Jia Li, Tomasyu Yu, Jiajin Li +5
Graph Neural Networks (GNNs) and Variational Autoencoders (VAEs) have been widely used in modeling and generating graphs with latent factors. However, there is no clear explanation…
Fast Epigraphical Projection-based Incremental Algorithms for Wasserstein Distributionally Robust Support Vector Machine
Jiajin Li, Caihua Chen, Anthony Man-Cho So
Wasserstein \textbf{D}istributionally \textbf{R}obust \textbf{O}ptimization (DRO) is concerned with finding decisions that perform well on data that are drawn from the worst-case p…
Understanding Notions of Stationarity in Non-Smooth Optimization
Jiajin Li, Anthony Man-Cho So, Wing-Kin Ma
Many contemporary applications in signal processing and machine learning give rise to structured non-convex non-smooth optimization problems that can often be tackled by simple ite…
The Gambler's Problem and Beyond
Baoxiang Wang, Shuai Li, Jiajin Li +1
We analyze the Gambler's problem, a simple reinforcement learning problem where the gambler has the chance to double or lose the bets until the target is reached. This is an early…
A First-Order Algorithmic Framework for Wasserstein Distributionally Robust Logistic Regression
Jiajin Li, Sen Huang, Anthony Man-Cho So
Wasserstein distance-based distributionally robust optimization (DRO) has received much attention lately due to its ability to provide a robustness interpretation of various learni…