166 citations · 182 across the 6 of their papers we have counts for
Showing 2025Show all
2 papers · 1 filter
cs.LG2025
CacheFormer: High Attention-Based Segment Caching
Sushant Singh, Ausif Mahmood
Efficiently handling long contexts in transformer-based language models with low perplexity is an active area of research. Numerous recent approaches like Linformer, Longformer, Pe…
cs.LG2025
Neural Attention: A Novel Mechanism for Enhanced Expressive Power in Transformer Models
Andrew DiGiugno, Ausif Mahmood
Transformer models typically calculate attention matrices using dot products, which have limitations when capturing nonlinear relationships between embedding vectors. We propose Ne…