2 papers
cs.LG2026
SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformers
Kiran Nair, Rodrigue Rizk, KC Santosh
Spiking neural networks (SNNs) offer an energy-efficient alternative to conventional deep neural networks by exploiting sparse event-driven computation, but their training remains…
cs.LG2026
CausalGate: Causal Importance Distillation for Transformer Module Pruning
Kiran Nair, Smriti Regmi, Rodrigue Rizk
Existing adaptive inference methods for Large Language Models rely on observational heuristics, such as hidden-state similarity or activation magnitudes, to drop redundant modules.…