2 papers
cs.LG2026
Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation
Hyunji Jung, Sungbin Shin, Namhoon Lee
Asynchronous pipeline parallelism maximizes hardware utilization by eliminating the pipeline bubbles inherent in synchronous execution, offering a path toward efficient large-scale…
cs.CL2024
Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization
Sungbin Shin, Wonpyo Park, Jaeho Lee +1
This work suggests fundamentally rethinking the current practice of pruning large language models (LLMs). The way it is done is by divide and conquer: split the model into submodel…