1 citations · 1 across the 7 of their papers we have counts for
7 papers
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
Zihan Qiu, Zekun Wang, Xiao Li +33
We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and additional 51B parameters of n…
On the Nonlinearity of Learning Rate Scaling for LLM Training
Zaiwen Yang, Huaqing Zhang, Jing Xu +1
Learning-rate transfer can reduce the cost of training large language models: instead of sweeping learning rates at target scale, practitioners extrapolate from smaller runs. Exist…
NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database
Weizhi Fei, Hao Shi, Jing Xu +7
Efficiently editing knowledge stored in large language models (LLMs) enables model updates without large-scale training. One possible solution is Locate-and-Edit (L\&E), allowing s…
Understanding Nonlinear Implicit Bias via Region Counts in Input Space
Jingwei Li, Jing Xu, Zifan Wang +2
One explanation for the strong generalization ability of neural networks is implicit bias. Yet, the definition and mechanism of implicit bias in non-linear contexts remains little…
Scalable Model Merging with Progressive Layer-wise Distillation
Jing Xu, Jiazheng Li, Jingzhao Zhang
Model merging offers an effective way to integrate the capabilities of multiple fine-tuned models. However, the performance degradation of the merged model remains a challenge, par…
Functionally Constrained Algorithm Solves Convex Simple Bilevel Problems
Huaqing Zhang, Lesi Chen, Jing Xu +1
This paper studies simple bilevel problems, where a convex upper-level function is minimized over the optimal solutions of a convex lower-level problem. We first show the fundament…
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
Jing Xu, Jingzhao Zhang
Fine-tuning large language models (LLM) can be costly. Parameter-efficient fine-tuning (PEFT) addresses the problems by training a fraction of the parameters, whose success reveals…