32 citations · 39 across the 6 of their papers we have counts for
6 papers
Delving into Muon and Beyond: Deep Analysis and Extensions
Xianbiao Qi, Marco Chen, Jiaquan Ye +2
The Muon optimizer has recently attracted considerable attention for its strong empirical performance and use of orthogonalized updates on matrix-shaped parameters, yet its underly…
SimpleGPT: Improving GPT via A Simple Normalization Strategy
Marco Chen, Xianbiao Qi, Yelin He +2
In this work, we revisit Transformer optimization through the lens of second-order geometry and establish a direct connection between architectural design, activation scale, the He…
DNT: a Deeply Normalized Transformer that can be trained by Momentum SGD
Xianbiao Qi, Marco Chen, Wenjie Xiao +4
Transformers have become the de facto backbone of modern deep learning, yet their training typically demands an advanced optimizer with adaptive learning rate like AdamW, rather th…
Taming Transformer Without Using Learning Rate Warmup
Xianbiao Qi, Yelin He, Jiaquan Ye +5
Scaling Transformer to a large scale without using some technical tricks such as learning rate warump and using an obviously lower learning rate is an extremely challenging task, a…
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML
Jiaquan Ye, Xianbiao Qi, Yelin He +4
This paper presents our solution for ICDAR 2021 competition on scientific literature parsing taskB: table recognition to HTML. In our method, we divide the table content recognitio…
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Table Image Recognition to Latex
Yelin He, Xianbiao Qi, Jiaquan Ye +5
This paper presents our solution for the ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX. This competition has two sub-tasks: Table Structure Reconstruction (…