15 citations · 34 across the 12 of their papers we have counts for
7 papers · 1 filter
Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers
Zixuan Jiang, Jiaqi Gu, Hanqing Zhu +1
Transformers have achieved great success in machine learning applications. Normalization techniques, such as Layer Normalization (LayerNorm, LN) and Root Mean Square Normalization…
Delving into Effective Gradient Matching for Dataset Condensation
Zixuan Jiang, Jiaqi Gu, Mingjie Liu +1
As deep learning models and datasets rapidly scale up, network training is extremely time-consuming and resource-costly. Instead of training on the entire dataset, learning with a…
L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization
Jiaqi Gu, Hanqing Zhu, Chenghao Feng +3
Silicon-photonics-based optical neural network (ONN) is a promising hardware platform that could represent a paradigm shift in efficient AI with its CMOS-compatibility, flexibility…
Delving into Macro Placement with Reinforcement Learning
Zixuan Jiang, Ebrahim Songhori, Shen Wang +5
In physical design, human designers typically place macros via trial and error, which is a Markov decision process. Reinforcement learning (RL) methods have demonstrated superhuman…
Towards Memory-Efficient Neural Networks via Multi-Level in situ Generation
Jiaqi Gu, Hanqing Zhu, Chenghao Feng +4
Deep neural networks (DNN) have shown superior performance in a variety of tasks. As they rapidly evolve, their escalating computation and memory demands make it challenging to dep…
Optimizer Fusion: Efficient Training with Better Locality and Parallelism
Zixuan Jiang, Jiaqi Gu, Mingjie Liu +2
Machine learning frameworks adopt iterative optimizers to train neural networks. Conventional eager execution separates the updating of trainable parameters from forward and backwa…
Logic Synthesis Meets Machine Learning: Trading Exactness for Generalization
Shubham Rai, Walter Lau Neto, Yukio Miyasaka +37
Logic synthesis is a fundamental step in hardware design whose goal is to find structural representations of Boolean functions while minimizing delay and area. If the function is c…