Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
Component Ablation for Efficient Hybrid Language Model Architectures: Performance, Resilience, and Compression Implications
Hector Borobia, Elies SeguÃ-Mas, Guillermina Tormo-Carbó
Hybrid language models combine softmax attention with linear-time sequence mechanisms such as state-space or linear-attention layers, but the functional contribution of each compon…
cs.CL2026
Component-Aware Self-Speculative Decoding in Hybrid Language Models
Hector Borobia, Elies SeguÃ-Mas, Guillermina Tormo-Carbó
Speculative decoding accelerates autoregressive inference by drafting candidate tokens with a fast model and verifying them in parallel with the target. Self-speculative methods av…
cs.CL2026
Where Should LoRA Go? Component-Type Placement in Hybrid Language Models
Hector Borobia, Elies SeguÃ-Mas, Guillermina Tormo-Carbó
Hybrid language models that interleave attention with recurrent components are increasingly competitive with pure Transformers, yet standard LoRA practice applies adapters uniforml…