26 citations · 50 across the 9 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
DeMo: Decoupled Momentum Optimization
Bowen Peng, Lizhang Chen, Baiyu Su +3
Scaling neural network training increasingly depends on synchronous data-parallelism, yet full-precision gradient all-reduce imposes a severe communication bottleneck. We propose D…
cs.CL2024★ 6 cited
Hermes 3 Technical Report
Ryan Teknium, Jeffrey Quesnelle, Chen Guang
Instruct (or "chat") tuned models have become the primary way in which most people interact with large language models. As opposed to "base" or "foundation" models, instruct-tuned…