3 papers
cs.CL2026
Magnitude Profile Pruning: Calibration-Free Structured Attention Head Removal for Transformer Compression
Kasun Dewage, Marianna Pensky, Heranga K. Rathnasekara +1
Structured pruning of attention heads provides a hardware-friendly way to compress Transformer language models. However, existing methods for measuring head-level importance requir…
cs.LG2026
Component Type, Not Reconstruction Error, Predicts Attention Quantization Sensitivity
Kasun Dewage, Marianna Pensky, Suranadi De Silva
Many post-training quantization (PTQ) methods use layer-wise reconstruction, second-order proxy objectives, or activation-aware transformations to reduce quantization-induced error…
cs.LG2026
LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights
Kasun Dewage, Marianna Pensky, Suranadi De Silva +1
We introduce LoRA-CRAFT (\textbf{C}ross-layer \textbf{R}ank \textbf{A}daptation via \textbf{F}rozen \textbf{T}ucker), abbreviated CRAFT throughout, an extremely parameter-efficient…