#residual stream expansion

topicresidual stream expansion

1 paper · 1 filter

cs.LG2026

xHC: Expanded Hyper-Connections

Xiangdong Zhang, Xiaohan Qin, Sunan Zou +10

The paper introduces xHC, a method that expands the residual stream of Transformers to many parallel streams using temporal feature augmentation and a sparse update scheme, enablin…