4 papers
Elucidating Representation Degradation Problem in Diffusion Model Training
Zhipeng Yao, Dazhou Li, Zitong Zhang +6
Diffusion models have achieved remarkable success, yet their training remains inefficient due to a severe optimization bottleneck, which we term Representation Degradation. As nois…
Dynamic Momentum Recalibration in Online Gradient Learning
Zhipeng Yao, Rui Yu, Guisong Chang +3
Stochastic Gradient Descent (SGD) and its momentum variants form the backbone of deep learning optimization, yet the underlying dynamics of their gradient behavior remain insuffici…
UDQL: Bridging The Gap between MSE Loss and The Optimal Value Function in Offline Reinforcement Learning
Yu Zhang, Rui Yu, Zhipeng Yao +3
The Mean Square Error (MSE) is commonly utilized to estimate the solution of the optimal value function in the vast majority of offline reinforcement learning (RL) models and has a…
Signal Processing Meets SGD: From Momentum to Filter
Zhipeng Yao, Rui Yu, Guisong Chang +3
In deep learning, stochastic gradient descent (SGD) and its momentum-based variants are widely used for optimization. However, the internal dynamics of these methods remain underex…