1 citations · 1 across the 4 of their papers we have counts for
4 papers
Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems
Yujun Kim, Jaeyoung Cha, Chulhee Yun
Recent theoretical results demonstrate that the convergence rates of permutation-based SGD (e.g., random reshuffling SGD) are faster than uniform-sampling SGD; however, these studi…
Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
Hanseul Cho, Jaeyoung Cha, Srinadh Bhojanapalli +1
Transformers often struggle with length generalization, meaning they fail to generalize to sequences longer than those encountered during training. While arithmetic tasks are commo…
Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
Hanseul Cho, Jaeyoung Cha, Pranjal Awasthi +3
Even for simple arithmetic tasks like integer addition, it is challenging for Transformers to generalize to longer sequences than those encountered during training. To tackle this…
Tighter Lower Bounds for Shuffling SGD: Random Permutations and Beyond
Jaeyoung Cha, Jaewook Lee, Chulhee Yun
We study convergence lower bounds of without-replacement stochastic gradient descent (SGD) for solving smooth (strongly-)convex finite-sum minimization problems. Unlike most existi…