1 citations · 2 across the 5 of their papers we have counts for
5 papers
Measuring Maximum Activations in Open Large Language Models
Luxuan Chen, Han Tian, Xinran Chen +9
The dynamic range of activations is a first-order constraint for low-bit quantization, activation scaling, and stable LLM inference. Prior work characterized outlier features and m…
EndPrompt: Efficient Long-Context Extension via Terminal Anchoring
Han Tian, Luxuan Chen, Xinran Chen +10
Extending the context window of large language models typically requires training on sequences at the target length, incurring quadratic memory and computational costs that make lo…
On the Performance and Memory Footprint of Distributed Training: An Empirical Study on Transformers
Zhengxian Lu, Fangyu Wang, Zhiwei Xu +2
Transformer models have emerged as potent solutions to a wide array of multidisciplinary challenges. The deployment of Transformer architectures is significantly hindered by their…
FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantization
Yi Zhang, Fei Yang, Shuang Peng +2
Large language models (LLMs) have demonstrated state-of-the-art performance across various tasks. However, the latency of inference and the large GPU memory consumption of LLMs res…
Holmes: Towards Distributed Training Across Clusters with Heterogeneous NIC Environment
Fei Yang, Shuang Peng, Ning Sun +5
Large language models (LLMs) such as GPT-3, OPT, and LLaMA have demonstrated remarkable accuracy in a wide range of tasks. However, training these models can incur significant expe…