1 citations · 1 across the 5 of their papers we have counts for
7 papers
SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training
Zhouyang Li, Yuliang Liu, Wei Zhang +4
Pipeline Parallelism (PP) serves as a crucial technique for training Large Language Models (LLMs), owing to its capability to alleviate memory pressure from model states with relat…
Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs
Chenxi Sun, Hongzhi Zhang, Zijia Lin +8
Large language models have demonstrated exceptional capability in natural language understanding and generation. However, their generation speed is limited by the inherently sequen…
KwaiYiiMath: Technical Report
Jiayi Fu, Lei Lin, Xiaoyang Gao +18
Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on math…
ASP: Automatic Selection of Proxy dataset for efficient AutoML
Peng Yao, Chao Liao, Jiyuan Jia +4
Deep neural networks have gained great success due to the increasing amounts of data, and diverse effective neural network designs. However, it also brings a heavy computing burden…
USDC: Unified Static and Dynamic Compression for Visual Transformer
Huan Yuan, Chao Liao, Jianchao Tan +5
Visual Transformers have achieved great success in almost all vision tasks, such as classification, detection, and so on. However, the model complexity and the inference speed of t…
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization
Yang Jin, Kun Xu, Liwei Chen +12
Recently, the remarkable advance of the Large Language Model (LLM) has inspired researchers to transfer its extraordinary reasoning capability to both vision and language data. How…