15 citations · 20 across the 11 of their papers we have counts for
11 papers
When Do Larger Batches Help Scale LLM Reinforcement Learning?
Ziniu Li, Jinbo Wang, Guanhua Huang +3
Larger batches reduce the variance of stochastic gradients per update and are therefore often expected to accelerate training. Yet whether this statistical benefit translates into…
Complementary Quantum Correlations Are Universal for Qubits
Jinbo Wang, Qihang Wang, Kun Chen
Extracting total correlations from a quantum system usually requires reconstructing its state, whereas many experiments access only a few measurement settings. A possible shortcut…
When Complementary Measurements Count the Same Classical Bit Twice: Counterexamples to CQC, ECQC, and Complementarity-Based Certification
Jinbo Wang, Qihang Wang, Kun Chen
Mutually unbiased measurements are commonly expected to expose independent facets of a quantum state: a correlation that is classical in one basis should disappear in a complementa…
SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering?
Yuxuan Sun, Yuze Zhao, Yufeng Wang +6
Evaluating software engineering capabilities has become a core component of modern large language models (LLMs); however, the key bottleneck hindering further scaling lies not in t…
More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations
Mingze Wang, Jinbo Wang, Yikuan Xia +2
Feedforward network (FFN) layers account for a large fraction of parameters and nonlinear expressivity in Transformer-based large language models (LLMs). Despite the evolution from…
Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Laws
Jinbo Wang, Binghui Li, Zhanpeng Zhou +5
Batch size scheduling (BSS) plays a critical role in large-scale deep learning training, influencing both optimization dynamics and computational efficiency. Yet, its theoretical f…
GradPower: Powering Gradients for Faster Language Model Pre-Training
Jinbo Wang, Mingze Wang, Jiaqi Zhang +5
We propose GradPower, a lightweight gradient-transformation technique for accelerating language model pre-training. Given a gradient vector , GradPower first applies the…
The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training
Jinbo Wang, Mingze Wang, Zhanpeng Zhou +3
Transformers consist of diverse building blocks, such as embedding layers, normalization layers, self-attention mechanisms, and point-wise feedforward networks. Thus, understanding…
: Language Modeling with Explicit Memory
Hongkang Yang, Zehao Lin, Wenjin Wang +13
The training and inference of large language models (LLMs) are together a costly process that transports knowledge from raw data to meaningful computation. Inspired by the memory h…
Improving Generalization and Convergence by Enhancing Implicit Regularization
Mingze Wang, Jinbo Wang, Haotian He +6
In this work, we propose an Implicit Regularization Enhancement (IRE) framework to accelerate the discovery of flat solutions in deep learning, thereby improving generalization and…
Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Study
Zeping Min, Jinbo Wang
This paper explores the integration of Large Language Models (LLMs) into Automatic Speech Recognition (ASR) systems to improve transcription accuracy. The increasing sophistication…