CausalGate: Causal Importance Distillation for Transformer Module Pruning

Kiran Nair, Smriti Regmi, Rodrigue Rizk

Existing adaptive inference methods for Large Language Models rely on observational heuristics, such as hidden-state similarity or activation magnitudes, to drop redundant modules.…