Component Ablation for Efficient Hybrid Language Model Architectures: Performance, Resilience, and Compression Implications

Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-Carbó

Hybrid language models combine softmax attention with linear-time sequence mechanisms such as state-space or linear-attention layers, but the functional contribution of each compon…