works on
efficient training 1large language models 1residual stream expansion 1sparse computation 1transformer architectures 1

From the 1 of 1 linked paper with an AI index.

collaborators

1 paper

cs.LG2026

xHC: Expanded Hyper-Connections

Xiangdong Zhang, Xiaohan Qin, Sunan Zou +10

The paper introduces xHC, a method that expands the residual stream of Transformers to many parallel streams using temporal feature augmentation and a sparse update scheme, enablin…