2 papers
cs.LG2024
Introducing Spectral Attention for Long-Range Dependency in Time Series Forecasting
Bong Gyun Kang, Dongjun Lee, HyunGi Kim +2
Sequence modeling faces challenges in capturing long-range dependencies across diverse tasks. Recent linear and transformer-based forecasters have shown superior performance in tim…
cs.LG2024
Grokfast: Accelerated Grokking by Amplifying Slow Gradients
Jaerin Lee, Bong Gyun Kang, Kihoon Kim +1
One puzzling artifact in machine learning dubbed grokking is where delayed generalization is achieved tenfolds of iterations after near perfect overfitting to the training data. Fo…