4 papers · 1 filter
Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
Xianzhen Luo, Yixuan Wang, Qingfu Zhu +4
Massive parameters of LLMs have made inference latency a fundamental bottleneck. Speculative decoding represents a lossless approach to accelerate inference through a guess-and-ver…
Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts
Xianzhen Luo, Qingfu Zhu, Zhiming Zhang +5
Program of Thoughts (PoT) is an approach characterized by its executable intermediate steps, which ensure the accuracy of the logical calculations in the reasoning process. Current…
Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training
Yixuan Wang, Xianzhen Luo, Fuxuan Wei +6
Existing speculative decoding methods typically require additional model structure and training processes to assist the model for draft token generation. This makes the migration o…
Semi-Instruct: Bridging Natural-Instruct and Self-Instruct for Code Large Language Models
Xianzhen Luo, Qingfu Zhu, Zhiming Zhang +4
Instruction tuning plays a pivotal role in Code Large Language Models (Code LLMs) for the task of program synthesis. Presently, two dominant paradigms for collecting tuning data ar…