5 citations · 6 across the 10 of their papers we have counts for
6 papers · 1 filter
Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives
Zecheng Wang, Deyuan Liu, Chunshan Li +5
Standard negative log-likelihood (NLL) for Supervised Fine-Tuning (SFT) applies uniform token-level weighting. This rigidity creates a two-fold failure mode: (i) overemphasizing lo…
Beyond Confidence: The Rhythms of Reasoning in Generative Models
Deyuan Liu, Zecheng Wang, Zhanyue Qin +3
Large Language Models (LLMs) exhibit impressive capabilities yet suffer from sensitivity to slight input context variations, hampering reliability. Conventional metrics like accura…
Surrogate Signals from Format and Length: Reinforcement Learning for Solving Mathematical Problems without Ground Truth Answers
Rihui Xin, Han Liu, Zecheng Wang +4
Large Language Models (LLMs) have achieved remarkable success in natural language processing tasks, with Reinforcement Learning (RL) playing a key role in adapting them to specific…
Baichuan-M1: Pushing the Medical Capability of Large Language Models
Bingning Wang, Haizhou Zhao, Huozhi Zhou +39
The current generation of large language models (LLMs) is typically designed for broad, general-purpose applications, while domain-specific LLMs, especially in vertical fields like…
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging
Deyuan Liu, Zhanyue Qin, Hairu Wang +12
While large language models (LLMs) excel in many domains, their complexity and scale challenge deployment in resource-limited environments. Current compression techniques, such as…
Checkpoint Merging via Bayesian Optimization in LLM Pretraining
Deyuan Liu, Zecheng Wang, Bingning Wang +6
The rapid proliferation of large language models (LLMs) such as GPT-4 and Gemini underscores the intense demand for resources during their training processes, posing significant ch…