70 citations · 205 across the 31 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2024★ 1 cited
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs
Peijie Dong, Lujun Li, Yuedong Zhong +8
In this paper, we present the first structural binarization method for LLM compression to less than 1-bit precision. Although LLMs have achieved remarkable performance, their memor…
cs.LG2024★ 4 cited
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models
Peijie Dong, Lujun Li, Zhenheng Tang +4
Despite the remarkable capabilities, Large Language Models (LLMs) face deployment challenges due to their extensive size. Pruning methods drop a subset of weights to accelerate, bu…
cs.LG2019★ 14 cited
Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Shaohuai Shi, Zhenheng Tang, Qiang Wang +2
To reduce the long training time of large deep neural network (DNN) models, distributed synchronous stochastic gradient descent (S-SGD) is commonly used on a cluster of workers. Ho…