7 citations · 9 across the 5 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Can an MLP Absorb Its Own Skip Connection Exactly?
Antonij Mijoski, Marko Karbevski
The benefits usually attributed to skip connections are optimization-theoretic: a smoother loss landscape and better gradient propagation. We ask a representational question instea…
cs.LG2026
Beyond Linearity in Attention Projections: The Case for Nonlinear Queries
Marko Karbevski
Recent algebraic analysis shows that in decoder-only and encoder-only transformers, the Query projection may be set to identity without noticeable performance deterioration.…