5 papers
Consolidator: Learning Persistent Routed Memory Across Context Boundaries
Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung
Copying short-term memory (STM) into a slower store can preserve state across a context boundary, but persistence alone does not ensure that the retained state influences subsequen…
Phasor Attention: Mean Root Square Normalization for Phase Manifold Preservation
Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung
While Root Mean Square Normalization has become the de facto standard for accelerating modern sequence models, its reliance on the quadratic accumulation of independent scalars ($\…
Z-Plane Neural Networks: Bounded Geometric Activation Replaces ReLU and LayerNorm
Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung
Modern deep neural networks rely on Euclidean scalar activations (e.g., ReLU) and global normalization techniques (e.g., LayerNorm) to prevent gradient instability in deep architec…
Phasor Memory Networks: Stable Backpropagation Through Time for Scalable Explicit Memory
Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung
For over a decade, explicit memory architectures like the Neural Turing Machine have remained theoretically appealing yet practically intractable for language modeling due to catas…
CaliciBoost: Performance-Driven Evaluation of Molecular Representations for Caco-2 Permeability Prediction
Huong Van Le, Weibin Ren, Junhong Kim +8
Caco-2 permeability serves as a critical in vitro indicator for predicting the oral absorption of drug candidates during early-stage drug discovery. To enhance the accuracy and eff…