8 citations · 13 across the 4 of their papers we have counts for
3 papers · 1 filter
Comparative Study of Large Language Model Architectures on Frontier
Junqi Yin, Avishek Bose, Guojing Cong +2
Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the d…
Optimizing Distributed Training on Frontier for Large Language Models
Sajal Dash, Isaac Lyngaas, Junqi Yin +5
Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Recent studies on loss sca…
Ultra-Long Sequence Distributed Transformer
Xiao Wang, Isaac Lyngaas, Aristeidis Tsaris +7
Transformer models trained on long sequences often achieve higher accuracy than short sequences. Unfortunately, conventional transformers struggle with long sequence training due t…