Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
On the Design of Hyper-Connections: Simplification, Stability, and Scalability
Xiangdong Zhang, Xiaohan Qin, Sunan Zou +8
Hyper-Connections (HC) architectures are increasingly adopted in frontier language models, yet their components remain insufficiently understood and introduce non-negligible traini…
cs.LG2026
xHC: Expanded Hyper-Connections
Xiangdong Zhang, Xiaohan Qin, Sunan Zou +10
Hyper-Connections (HC) expand the residual stream of Transformers into parallel streams, providing a form of memory scaling beyond model width and depth. Manifold-Constrained H…
cs.LG2026
BigMac: Breaking the Pareto Frontier of Compute and Memory in Multimodal LLM Training
Zili Zhang, Chengxu Yang, Shenglong Zhang +8
Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity. Existing systems redesign the training pipeline to address these challenges, b…