2 citations · 2 across the 6 of their papers we have counts for
4 papers · 1 filter
ReGLA: Refining Gated Linear Attention
Peng Lu, Ivan Kobyzev, Mehdi Rezagholizadeh +2
Recent advancements in Large Language Models (LLMs) have set themselves apart with their exceptional performance in complex language modelling tasks. However, these models are also…
Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similarity
Michael R. Metel, Peng Lu, Boxing Chen +2
We present a simple on the fly method for faster inference of large language models. Unlike other (self-)speculative decoding techniques, our method does not require fine-tuning or…
Resonance RoPE: Improving Context Length Generalization of Large Language Models
Suyuchen Wang, Ivan Kobyzev, Peng Lu +2
This paper addresses the challenge of train-short-test-long (TSTL) scenarios in Large Language Models (LLMs) equipped with Rotary Position Embedding (RoPE), where models pre-traine…
Hyperparameter Optimization for Large Language Model Instruction-Tuning
Christophe Tribes, Sacha Benarroch-Lelong, Peng Lu +1
The fine-tuning of Large Language Models (LLMs) has enabled them to recently achieve milestones in natural language processing applications. The emergence of ever larger LLMs has p…