204 citations · 213 across the 16 of their papers we have counts for
6 papers · 1 filter
GPT Carry-On: Training Foundation Model for Customization Could Be Simple, Scalable and Affordable
Jianqiao Wangni
Modern large language foundation models (LLM) have now entered the daily lives of millions of users. We ask a natural question whether it is possible to customize LLM for every use…
Quantized Adaptive Subgradient Algorithms and Their Applications
Ke Xu, Jianqiao Wangni, Yifan Zhang +3
Data explosion and an increase in model size drive the remarkable advances in large-scale machine learning, but also make model training time-consuming and model storage difficult.…
Trajectory Normalized Gradients for Distributed Optimization
Jianqiao Wangni, Ke Li, Jianbo Shi +1
Recently, researchers proposed various low-precision gradient compression, for efficient communication in large-scale distributed optimization. Based on these work, we try to reduc…
Learning Sparse Visual Representations with Leaky Capped Norm Regularizers
Jianqiao Wangni, Dahua Lin
Sparsity inducing regularization is an important part for learning over-complete visual representations. Despite the popularity of regularization, in this paper, we invest…
Gradient Sparsification for Communication-Efficient Distributed Optimization
Jianqiao Wangni, Jialei Wang, Ji Liu +1
Modern large scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures. A key bottleneck is the co…
Training L1-Regularized Models with Orthant-Wise Passive Descent Algorithms
Jianqiao Wangni
The -regularized models are widely used for sparse regression or classification tasks. In this paper, we propose the orthant-wise passive descent algorithm (OPDA) for optimizi…