PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning

Yazdan Jamshidi, Alexey Shvets

One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Perc…