3 citations · 4 across the 8 of their papers we have counts for
10 papers
Robust Langevin Sampling under State-Dependent Heavy-Tailed Noise with Applications to LLM Fine-Tuning
M. Berk Sahin, Ege C. Kaya, Arda Fazla +2
Existing convergence guarantees for non-log-concave sampling with stochastic gradient Langevin Monte Carlo (SG-LMC) typically assume globally Lipschitz gradients and uniformly boun…
Why Does Adaptive Batching Help LLM Pretraining? A Perspective from Unbounded Variance
Arda Fazla, Antesh Upadhyay, Ege C. Kaya +2
Increasing the batch size during training is a common practice in large language model (LLM) pretraining, yet the theoretical justification behind its success is not well understoo…
A Banach-Space Theory of Markovian Halpern Iteration for Non-Expansive Maps
Ege C. Kaya, Arda Fazla, M. Berk Sahin +1
We study stochastic approximation of fixed points of a non-expansive operator when the oracle samples originate from a continuing Markovian trajectory. A direct block-minibatch…
Mitigating Spurious Correlations with Memorization-Guided Dataset De-Biasing
Arda Fazla, Abolfazl Hashemi
Real-world datasets often contain spurious correlations that are not causally related to the target label. When such correlations dominate the majority of training samples, models…
Beyond Bounded Variance: Variance-Reduced Normalized Methods for Nonconvex Optimization under Blum-Gladyshev Noise
Antesh Upadhyay, Arda Fazla, Abolfazl Hashemi
We study nonconvex stochastic optimization under the Blum-Gladyshev (-0) noise model, where the stochastic gradient variance grows quadratically with the distance from…
Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance
Arda Fazla, Ege C. Kaya, Antesh Upadhyay +1
Analysis of Stochastic Gradient Descent (SGD) and its variants typically relies on the assumption of uniformly bounded variance, a condition that frequently fails in practical non-…
Hybrid State Space-based Learning for Sequential Data Prediction with Joint Optimization
Mustafa E. Aydın, Arda Fazla, Suleyman S. Kozat
We investigate nonlinear prediction/regression in an online setting and introduce a hybrid model that effectively mitigates, via a joint mechanism through a state space formulation…
Context-Aware Ensemble Learning for Time Series
Arda Fazla, Mustafa Enes Aydin, Orhun Tamyigit +1
We investigate ensemble methods for prediction in an online setting. Unlike all the literature in ensembling, for the first time, we introduce a new approach using a meta learner t…
Numerical Weather Forecasting using Convolutional-LSTM with Attention and Context Matcher Mechanisms
Selim Furkan Tekin, Arda Fazla, Suleyman Serdar Kozat
Numerical weather forecasting using high-resolution physical models often requires extensive computational resources on supercomputers, which diminishes their wide usage in most re…
Semi-Automatic Annotation For Visual Object Tracking
Kutalmis Gokalp Ince, Aybora Koksal, Arda Fazla +1
We propose a semi-automatic bounding box annotation method for visual object tracking by utilizing temporal information with a tracking-by-detection approach. For detection, we use…