13 citations · 38 across the 12 of their papers we have counts for
9 papers · 1 filter
Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers
Zixuan Jiang, Jiaqi Gu, Hanqing Zhu +1
Transformers have achieved great success in machine learning applications. Normalization techniques, such as Layer Normalization (LayerNorm, LN) and Root Mean Square Normalization…
HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression
Jiaqi Gu, Ben Keller, Jean Kossaifi +3
Transformers have attained superior performance in natural language processing and computer vision. Their self-attention and feedforward layers are overparameterized, limiting infe…
An Adversarial Active Sampling-based Data Augmentation Framework for Manufacturable Chip Design
Mingjie Liu, Haoyu Yang, Zongyi Li +7
Lithography modeling is a crucial problem in chip design to ensure a chip design mask is manufacturable. It requires rigorous simulations of optical and chemical models that are co…
L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization
Jiaqi Gu, Hanqing Zhu, Chenghao Feng +3
Silicon-photonics-based optical neural network (ONN) is a promising hardware platform that could represent a paradigm shift in efficient AI with its CMOS-compatibility, flexibility…
DNN-Opt: An RL Inspired Optimization for Analog Circuit Sizing using Deep Neural Networks
Ahmet F. Budak, Prateek Bhansali, Bo Liu +3
Analog circuit sizing takes a significant amount of manual effort in a typical design cycle. With rapidly developing technology and tight schedules, bringing automated solutions fo…
Towards Memory-Efficient Neural Networks via Multi-Level in situ Generation
Jiaqi Gu, Hanqing Zhu, Chenghao Feng +4
Deep neural networks (DNN) have shown superior performance in a variety of tasks. As they rapidly evolve, their escalating computation and memory demands make it challenging to dep…