Is SwiGLU's Open Positive Tail Necessary? Evidence from Closed-Tail Gating with MemGLU

Yuting Ge, Pengju Yang, Mingkai Nie

We test whether decoder-only language-model FFNs require SwiGLU's open positive tail. We introduce MemGLU as a closed-tail comparator derived from a memristive branch geometry. Acr…