21 citations · 21 across the 3 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
On the Design of Hyper-Connections: Simplification, Stability, and Scalability
Xiangdong Zhang, Xiaohan Qin, Sunan Zou +8
Hyper-Connections (HC) architectures are increasingly adopted in frontier language models, yet their components remain insufficiently understood and introduce non-negligible traini…
cs.LG2026
xHC: Expanded Hyper-Connections
Xiangdong Zhang, Xiaohan Qin, Sunan Zou +10
Hyper-Connections (HC) expand the residual stream of Transformers into parallel streams, providing a form of memory scaling beyond model width and depth. Manifold-Constrained H…
cs.LG2025
UltraSketchLLM: Sub-1-Bit LLM Compression via Sketch and Hardware-Friendly Operators
Sunan Zou, Xueting Sun, Ziyun Zhang +1
Large language models (LLMs) require larger GPU memory size these days, necessitating efficient and extreme weight compression methods. Existing compression methods are either theo…