71 citations · 558 across the 67 of their papers we have counts for
69 papers
Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping
Umut Şimşekli, Mert Gürbüzbalaban, Sinan Yıldırım +1
The injection of heavy-tailed noise into the iterates of stochastic gradient descent (SGD) has garnered growing interest in recent years due to its theoretical and empirical benefi…
Short-maturity asymptotics for option prices with interest rates effects
Dan Pirjol, Lingjiong Zhu
We derive the short-maturity asymptotics for option prices in the local volatility model in a new short-maturity limit at fixed , where is the interest rat…
Asymptotics for Short Maturity Asian Options in Jump-Diffusion models with Local Volatility
Dan Pirjol, Lingjiong Zhu
We present a study of the short maturity asymptotics for Asian options in a jump-diffusion model with a local volatility component, where the jumps are modeled as a compound Poisso…
Fluctuations and moderate deviations for the mean fields of Hawkes processes
Fuqing Gao, Yunshi Gao, Lingjiong Zhu
The Hawkes process is a counting process that has self- and mutually-exciting features with many applications in various fields. In recent years, there have been many interests in…
Asymptotics for the Laplace transform of the time integral of the geometric Brownian motion
Dan Pirjol, Lingjiong Zhu
We present an asymptotic result for the Laplace transform of the time integral of the geometric Brownian motion with $X_T = \int_0^T e^{σW_s + ( a…
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent
Lingjiong Zhu, Mert Gurbuzbalaban, Anant Raj +1
Algorithmic stability is an important notion that has proven powerful for deriving generalization bounds for practical algorithms. The last decade has witnessed an increasing numbe…
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD than Constant Stepsize
Mert Gürbüzbalaban, Yuanhan Hu, Umut Şimşekli +1
Cyclic and randomized stepsizes are widely used in the deep learning practice and can often outperform standard stepsize choices such as constant stepsize in SGD. Despite their emp…
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions
Anant Raj, Lingjiong Zhu, Mert Gürbüzbalaban +1
Heavy-tail phenomena in stochastic gradient descent (SGD) have been reported in several empirical studies. Experimental evidence in previous works suggests a strong interplay betwe…
Sensitivities of Asian options in the Black-Scholes model
Dan Pirjol, Lingjiong Zhu
We propose analytical approximations for the sensitivities (Greeks) of the Asian options in the Black-Scholes model, following from a small maturity/volatility approximation for th…
A delayed dual risk model
Lingjiong Zhu
In this paper, we study a dual risk model with delays in the spirit of Dassios-Zhao. When a new innovation occurs, there is a delay before the innovation turns into a profit. We ob…
Large deviations for the mean-field limit of Hawkes processes
Fuqing Gao, Lingjiong Zhu
Hawkes processes are a class of simple point processes whose intensity depends on the past history, and is in general non-Markovian. Limit theorems for Hawkes processes in various…
Penalized Overdamped and Underdamped Langevin Monte Carlo Algorithms for Constrained Sampling
Mert Gürbüzbalaban, Yuanhan Hu, Lingjiong Zhu
We consider the constrained sampling problem where the goal is to sample from a target distribution when is constrained to lie on a convex body $\mathca…
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares
Anant Raj, Melih Barsbey, Mert Gürbüzbalaban +2
Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error. While these studies hav…
Heavy-Tail Phenomenon in Decentralized SGD
Mert Gurbuzbalaban, Yuanhan Hu, Umut Simsekli +2
Recent theoretical studies have shown that heavy-tails can emerge in stochastic optimization due to `multiplicative noise', even under surprisingly simple settings, such as linear…
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms
Alexander Camuto, George Deligiannidis, Murat A. Erdogdu +3
Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the d…
Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance
Hongjian Wang, Mert Gürbüzbalaban, Lingjiong Zhu +2
Recent studies have provided both empirical and theoretical evidence illustrating that heavy tails can emerge in stochastic gradient descent (SGD) in various scenarios. Such heavy…
Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections
Alexander Camuto, Xiaoyu Wang, Lingjiong Zhu +3
Gaussian noise injections (GNIs) are a family of simple and widely-used regularisation methods for training neural networks, where one injects additive or multiplicative Gaussian n…
Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
Mert Gürbüzbalaban, Xuefeng Gao, Yuanhan Hu +1
Stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Hamiltonian Monte Carlo (SGHMC) are two popular Markov Chain Monte Carlo (MCMC) algorithms for Bayesian inferen…
Non-Convex Optimization via Non-Reversible Stochastic Gradient Langevin Dynamics
Yuanhan Hu, Xiaoyu Wang, Xuefeng Gao +2
Stochastic Gradient Langevin Dynamics (SGLD) is a powerful algorithm for optimizing a non-convex objective, where a controlled and properly scaled Gaussian noise is added to the st…
The Heavy-Tail Phenomenon in SGD
Mert Gurbuzbalaban, Umut Şimşekli, Lingjiong Zhu
In recent years, various notions of capacity and complexity have been proposed for characterizing the generalization properties of stochastic gradient descent (SGD) in deep learnin…
Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise
Umut Şimşekli, Lingjiong Zhu, Yee Whye Teh +1
Stochastic gradient descent with momentum (SGDm) is one of the most popular optimization algorithms in deep learning. While there is a rich theory of SGDm for convex problems, the…
Asymptotics of the time-discretized log-normal SABR model: The implied volatility surface
Dan Pirjol, Lingjiong Zhu
We propose a novel time discretization for the log-normal SABR model which is a popular stochastic volatility model that is widely used in financial practice. Our time discretizati…
Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks
Alireza Fallah, Mert Gurbuzbalaban, Asuman Ozdaglar +2
We study distributed stochastic gradient (D-SG) method and its accelerated variant (D-ASG) for solving decentralized strongly convex stochastic optimization problems where the obje…
Explosion in the quasi-Gaussian HJM model
Dan Pirjol, Lingjiong Zhu
We study the explosion of the solutions of the SDE in the quasi-Gaussian HJM model with a CEV-type volatility. The quasi-Gaussian HJM models are a popular approach for modeling the…