most citedExtending Llama-3's Context Ten-Fold Overnight

4 citations · 4 across the 1 of their papers we have counts for

collaborators

1 paper

cs.CL2024★ 4 cited

Extending Llama-3's Context Ten-Fold Overnight

Peitian Zhang, Ninglu Shao, Zheng Liu +4

We extend the context length of Llama-3-8B-Instruct from 8K to 80K via QLoRA fine-tuning. The entire training cycle is super efficient, which takes 8 hours on one 8xA800 (80G) GPU…