387 citations · 2.2k across the 41 of their papers we have counts for
1 paper · 1 filter
Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano +2
Despite advances using low-rank adapters and quantization, pretraining of large models on consumer hardware has not been possible without model sharding, offloading during training…