Rethinking Learning Rate Tuning in the Era of Large Language Models

Hongpeng Jin, Wenqi Wei, Xuyu Wang +2

Large Language Models (LLMs) represent the recent success of deep learning in achieving remarkable human-like predictive performance. It has become a mainstream strategy to leverag…