182 citations · 364 across the 29 of their papers we have counts for
8 papers · 1 filter
Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers
Zixuan Jiang, Jiaqi Gu, Hanqing Zhu +1
Transformers have achieved great success in machine learning applications. Normalization techniques, such as Layer Normalization (LayerNorm, LN) and Root Mean Square Normalization…
HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression
Jiaqi Gu, Ben Keller, Jean Kossaifi +3
Transformers have attained superior performance in natural language processing and computer vision. Their self-attention and feedforward layers are overparameterized, limiting infe…
Delving into Effective Gradient Matching for Dataset Condensation
Zixuan Jiang, Jiaqi Gu, Mingjie Liu +1
As deep learning models and datasets rapidly scale up, network training is extremely time-consuming and resource-costly. Instead of training on the entire dataset, learning with a…
L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization
Jiaqi Gu, Hanqing Zhu, Chenghao Feng +3
Silicon-photonics-based optical neural network (ONN) is a promising hardware platform that could represent a paradigm shift in efficient AI with its CMOS-compatibility, flexibility…
QuantumNAT: Quantum Noise-Aware Training with Noise Injection, Quantization and Normalization
Hanrui Wang, Jiaqi Gu, Yongshan Ding +4
Parameterized Quantum Circuits (PQC) are promising towards quantum advantage on near-term quantum hardware. However, due to the large quantum noises (errors), the performance of PQ…
Towards Memory-Efficient Neural Networks via Multi-Level in situ Generation
Jiaqi Gu, Hanqing Zhu, Chenghao Feng +4
Deep neural networks (DNN) have shown superior performance in a variety of tasks. As they rapidly evolve, their escalating computation and memory demands make it challenging to dep…
Optimizer Fusion: Efficient Training with Better Locality and Parallelism
Zixuan Jiang, Jiaqi Gu, Mingjie Liu +2
Machine learning frameworks adopt iterative optimizers to train neural networks. Conventional eager execution separates the updating of trainable parameters from forward and backwa…
Logic Synthesis Meets Machine Learning: Trading Exactness for Generalization
Shubham Rai, Walter Lau Neto, Yukio Miyasaka +37
Logic synthesis is a fundamental step in hardware design whose goal is to find structural representations of Boolean functions while minimizing delay and area. If the function is c…