3 papers
cs.CL2023★ 1 cited
One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning
Guangtao Zeng, Peiyuan Zhang, Wei Lu
Fine-tuning pre-trained language models for multiple tasks tends to be expensive in terms of storage. To mitigate this, parameter-efficient transfer learning (PETL) methods have be…
cs.LG2023
Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization
Peiyuan Zhang, Jiaye Teng, Jingzhao Zhang
This work studies the generalization error of gradient methods. More specifically, we focus on how training steps and step-size might affect generalization in smooth stocha…
math.OC2019
Mixing of Stochastic Accelerated Gradient Descent
Peiyuan Zhang, Hadi Daneshmand, Thomas Hofmann
We study the mixing properties for stochastic accelerated gradient descent (SAGD) on least-squares regression. First, we show that stochastic gradient descent (SGD) and SAGD are si…