efficient training 1large language models 1residual stream expansion 1sparse computation 1transformer architectures 1
From the 1 of 3 linked papers with an AI index.
3 papers
cs.LG2026
xHC: Expanded Hyper-Connections
Xiangdong Zhang, Xiaohan Qin, Sunan Zou +10
The paper introduces xHC, a method that expands the residual stream of Transformers to many parallel streams using temporal feature augmentation and a sparse update scheme, enablin…
cs.LG2026
UltraSketchLLM: Sub-1-Bit LLM Compression via Sketch and Hardware-Friendly Operators
Sunan Zou, Xueting Sun, Ziyun Zhang +1
Large language models (LLMs) require larger GPU memory size these days, necessitating efficient and extreme weight compression methods. Existing compression methods are either theo…
cs.AR2024
The Dawn of AI-Native EDA: Opportunities and Challenges of Large Circuit Models
Lei Chen, Yiqi Chen, Zhufei Chu +36
Within the Electronic Design Automation (EDA) domain, AI-driven solutions have emerged as formidable tools, yet they typically augment rather than redefine existing methodologies.…