2 papers
cs.CL2026
Avey-B
Devang Acharya, Mohammad Hammoud
Compact pretrained bidirectional encoders remain the backbone of industrial NLP under tight compute and memory budgets. Their effectiveness stems from self-attention's ability to d…
cs.CL2025
Don't Pay Attention
Mohammad Hammoud, Devang Acharya
The Transformer has become the de facto standard for modern language models owing to its parallelizable training and effective autoregressive decoding. However, its fixed context w…