3 papers
cs.MS2020
AutoHOOT: Automatic High-Order Optimization for Tensors
Linjian Ma, Jiayu Ye, Edgar Solomonik
High-order optimization methods, including Newton's method and its variants as well as alternating minimization methods, dominate the optimization algorithms for tensor decompositi…
cs.CL2019
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Sheng Shen, Zhen Dong, Jiayu Ye +5
Transformer based architectures have become de-facto models used for a range of Natural Language Processing tasks. In particular, the BERT based models achieved significant accurac…
cs.LG2019
Inefficiency of K-FAC for Large Batch Size Training
Linjian Ma, Gabe Montague, Jiayu Ye +4
In stochastic optimization, using large batch sizes during training can leverage parallel resources to produce faster wall-clock training times per training epoch. However, for bot…