5 papers
Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
Xianzhen Luo, Yixuan Wang, Qingfu Zhu +4
Massive parameters of LLMs have made inference latency a fundamental bottleneck. Speculative decoding represents a lossless approach to accelerate inference through a guess-and-ver…
Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training
Yixuan Wang, Xianzhen Luo, Fuxuan Wei +6
Existing speculative decoding methods typically require additional model structure and training processes to assist the model for draft token generation. This makes the migration o…
Semi-Instruct: Bridging Natural-Instruct and Self-Instruct for Code Large Language Models
Xianzhen Luo, Qingfu Zhu, Zhiming Zhang +4
Instruction tuning plays a pivotal role in Code Large Language Models (Code LLMs) for the task of program synthesis. Presently, two dominant paradigms for collecting tuning data ar…
Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts
Xianzhen Luo, Qingfu Zhu, Zhiming Zhang +5
Program of Thoughts (PoT) is an approach characterized by its executable intermediate steps, which ensure the accuracy of the logical calculations in the reasoning process. Current…
On Robust Wasserstein Barycenter: The Model and Algorithm
Xu Wang, Jiawei Huang, Qingyuan Yang +1
The Wasserstein barycenter problem is to compute the average of given probability measures, which has been widely studied in many different areas; however, real-world data sets…