15 citations · 59 across the 29 of their papers we have counts for
9 papers · 1 filter
ISO: An RLVR-Native Optimization Stack
Hanqing Zhu, Wenyan Cong, Zhizhou Sha +8
Reinforcement learning with verifiable rewards (RLVR) advances the reasoning capabilities of language models, yet the optimization layer that converts reward feedback into weight-s…
Geometry-Preserving Orthonormal Initialization for Low-Rank Adaptation in RLVR
Ruijia Zhang, Jiacheng Zhu, Hanqing Zhu +1
Low-rank adaptation (LoRA) and its variants enable parameter-efficient fine-tuning of large language models under the supervised fine-tuning (SFT) paradigm. However, their efficacy…
The Path Not Taken: RLVR Provably Learns Off the Principals
Hanqing Zhu, Zhenyu Zhang, Hanxian Huang +11
Reinforcement Learning with Verifiable Rewards (RLVR) reliably improves the reasoning performance of large language models, yet it appears to modify only a small fraction of parame…
APOLLO: SGD-like Memory, AdamW-level Performance
Hanqing Zhu, Zhenyu Zhang, Wenyan Cong +7
Large language models (LLMs) are notoriously memory-intensive during training, particularly with the popular AdamW optimizer. This memory burden necessitates using more or higher-e…
PACE: Pacing Operator Learning to Accurate Optical Field Simulation for Complicated Photonic Devices
Hanqing Zhu, Wenyan Cong, Guojin Chen +4
Electromagnetic field simulation is central to designing, optimizing, and validating photonic devices and circuits. However, costly computation associated with numerical simulation…
INSIGHT: Universal Neural Simulator for Analog Circuits Harnessing Autoregressive Transformers
Souradip Poddar, Youngmin Oh, Yao Lai +3
Analog front-end design heavily relies on specialized human expertise and costly trial-and-error simulations, which motivated many prior works on analog design automation. However,…
Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers
Zixuan Jiang, Jiaqi Gu, Hanqing Zhu +1
Transformers have achieved great success in machine learning applications. Normalization techniques, such as Layer Normalization (LayerNorm, LN) and Root Mean Square Normalization…
L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization
Jiaqi Gu, Hanqing Zhu, Chenghao Feng +3
Silicon-photonics-based optical neural network (ONN) is a promising hardware platform that could represent a paradigm shift in efficient AI with its CMOS-compatibility, flexibility…
Towards Memory-Efficient Neural Networks via Multi-Level in situ Generation
Jiaqi Gu, Hanqing Zhu, Chenghao Feng +4
Deep neural networks (DNN) have shown superior performance in a variety of tasks. As they rapidly evolve, their escalating computation and memory demands make it challenging to dep…