167 citations · 914 across the 103 of their papers we have counts for
10 papers · 1 filter
K-Forcing: Joint Next-K-Token Decoding via Push-Forward Language Modeling
Zhiwei Tang, Yuanyu He, Yizheng Han +4
Autoregressive (AR) language modeling is the dominant paradigm for text generation, yet its sequential token-by-token decoding makes inference memory-bound and inefficient. Existin…
R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning
Zhuokun Chen, Zeren Chen, Jiahao He +4
Chain-of-thought (CoT) enhances the problem-solving ability of large language models (LLMs) but incurs substantial inference cost due to long autoregressive trajectories. Existing…
Sparsity Forcing: Reinforcing Token Sparsity of MLLMs
Feng Chen, Yefei He, Lequan Lin +4
Sparse attention mechanisms aim to reduce computational overhead with minimal accuracy loss by selectively processing salient tokens. Despite their effectiveness, most methods mere…
ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification
Yefei He, Luoming Zhang, Weijia Wu +3
KV cache stores key and value states from previous tokens to avoid re-computation, yet it demands substantial storage space, especially for long sequences. Adaptive KV cache compre…
Efficient Stitchable Task Adaptation
Haoyu He, Zizheng Pan, Jing Liu +2
The paradigm of pre-training and fine-tuning has laid the foundation for deploying deep learning models. However, most fine-tuning methods are designed to meet a specific resource…
LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning
Mingyang Zhang, Hao Chen, Chunhua Shen +4
Large Language Models (LLMs), such as LLaMA and T5, have shown exceptional performance across various tasks through fine-tuning. Although low-rank adaption (LoRA) has emerged to ch…
Stitchable Neural Networks
Zizheng Pan, Jianfei Cai, Bohan Zhuang
The public model zoo containing enormous powerful pretrained model families (e.g., ResNet/DeiT) has reached an unprecedented scope than ever, which significantly contributes to the…
A Survey on Efficient Training of Transformers
Bohan Zhuang, Jing Liu, Zizheng Pan +3
Recent advances in Transformers have come with a huge requirement on computing resources, highlighting the importance of developing efficient training techniques to make Transforme…
FATNN: Fast and Accurate Ternary Neural Networks
Peng Chen, Bohan Zhuang, Chunhua Shen
Ternary Neural Networks (TNNs) have received much attention due to being potentially orders of magnitude faster in inference, as well as more power efficient, than full-precision c…
Role-Wise Data Augmentation for Knowledge Distillation
Jie Fu, Xue Geng, Zhijian Duan +6
Knowledge Distillation (KD) is a common method for transferring the ``knowledge'' learned by one machine learning model (the \textit{teacher}) into another model (the \textit{stude…