most citedOutlier-Efficient Hopfield Layers for Large Transformer-Based Models

2 citations · 2 across the 8 of their papers we have counts for

collaborators
Showing cs.LGShow all

2 papers · 1 filter

cs.LG2025

Universal Approximation with Softmax Attention

Jerry Yao-Chieh Hu, Hude Liu, Hong-Yu Chen +2

We prove that with linear transformations, both (i) two-layer self-attention and (ii) one-layer self-attention followed by a softmax function are universal approximators for contin…

cs.LG2024★ 2 cited

Outlier-Efficient Hopfield Layers for Large Transformer-Based Models

Jerry Yao-Chieh Hu, Pei-Hsuan Chang, Robin Luo +4

We introduce an Outlier-Efficient Modern Hopfield Model (termed ) and use it to address the outlier inefficiency problem of {training} gigantic transformer-base…