4 papers
UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming
Hao Lin, Ke Wu, Jie Li +2
Distributed learning is commonly used for training deep learning models, especially large models. In distributed learning, manual parallelism (MP) methods demand considerable human…
Multiple Code Hashing for Efficient Image Retrieval
Ming-Wei Li, Qing-Yuan Jiang, Wu-Jun Li
Due to its low storage cost and fast query speed, hashing has been widely used in large-scale image retrieval tasks. Hash bucket search returns data points within a given Hamming r…
On the Evaluation Metric for Hashing
Qing-Yuan Jiang, Ming-Wei Li, Wu-Jun Li
Due to its low storage cost and fast query speed, hashing has been widely used for large-scale approximate nearest neighbor (ANN) search. Bucket search, also called hash lookup, ca…
Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training
Shen-Yi Zhao, Chang-Wei Shi, Yin-Peng Xie +1
Stochastic gradient descent~(SGD) and its variants have been the dominating optimization methods in machine learning. Compared to SGD with small-batch training, SGD with large-batc…