1 citations · 1 across the 9 of their papers we have counts for
9 papers
Stronger Lower Bounds for (Non-)Anytime Acceleration of Gradient Descent
Minchan Jung, Hanseul Cho, Chulhee Yun
The rate-optimal convergence rate of gradient descent (GD) with a fixed step-size is well known to be for -Lipschitz smooth convex objectives in the prior art in con…
The Cost of Robustness: Tighter Bounds on Parameter Complexity for Robust Memorization in ReLU Nets
Yujun Kim, Chaewon Moon, Chulhee Yun
We study the parameter complexity of robust memorization for networks: the number of parameters required to interpolate any given dataset with -separation betwee…
Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems
Yujun Kim, Jaeyoung Cha, Chulhee Yun
Recent theoretical results demonstrate that the convergence rates of permutation-based SGD (e.g., random reshuffling SGD) are faster than uniform-sampling SGD; however, these studi…
Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification
Hyunji Jung, Hanseul Cho, Chulhee Yun
We study continual learning on multiple linear classification tasks by sequentially running gradient descent (GD) for a fixed budget of iterations per task. When all tasks are join…
Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
Hanseul Cho, Jaeyoung Cha, Srinadh Bhojanapalli +1
Transformers often struggle with length generalization, meaning they fail to generalize to sequences longer than those encountered during training. While arithmetic tasks are commo…
Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
Hanseul Cho, Jaeyoung Cha, Pranjal Awasthi +3
Even for simple arithmetic tasks like integer addition, it is challenging for Transformers to generalize to longer sequences than those encountered during training. To tackle this…
Fundamental Benefit of Alternating Updates in Minimax Optimization
Jaewook Lee, Hanseul Cho, Chulhee Yun
The Gradient Descent-Ascent (GDA) algorithm, designed to solve minimax optimization problems, takes the descent and ascent steps either simultaneously (Sim-GDA) or alternately (Alt…
Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults
Prin Phunyaphibarn, Junghyun Lee, Bohan Wang +2
Although gradient descent with Polyak's momentum is widely used in modern machine and deep learning, a concrete understanding of its effects on the training trajectory remains elus…
Fair Streaming Principal Component Analysis: Statistical and Algorithmic Viewpoint
Junghyun Lee, Hanseul Cho, Se-Young Yun +1
Fair Principal Component Analysis (PCA) is a problem setting where we aim to perform PCA while making the resulting representation fair in that the projected distributions, conditi…