2 papers
cs.LG2026
Ask Self, Ask Others: Relation Is All You Need
Yuting Ge, Pengju Yang, Mingkai Nie
Attention dominates token mixing, but it collapses relation formation and flow allocation into a single score-to-flow step. We introduce Relation, which separates them by first org…
cs.LG2026
Is SwiGLU's Open Positive Tail Necessary? Evidence from Closed-Tail Gating with MemGLU
Yuting Ge, Pengju Yang, Mingkai Nie
We test whether decoder-only language-model FFNs require SwiGLU's open positive tail. We introduce MemGLU as a closed-tail comparator derived from a memristive branch geometry. Acr…