4 papers · 1 filter
OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers
Siyuan Li, Jiabao Pan, Yumou Liu +9
Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the land…
Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
Chuxue Cao, Jinluan Yang, Haoran Li +8
Large Language Models (LLMs) show remarkable capabilities, yet their stochastic next-token prediction creates logical inconsistencies and reward hacking that formal symbolic system…
ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoning
Qizhi Pei, Zhuoshi Pan, Honglin Lin +6
Large Reasoning Models (LRMs) have shown impressive capabilities in complex problem-solving, often benefiting from training on difficult mathematical problems that stimulate intric…
LEMMA: Learning from Errors for MatheMatical Advancement in LLMs
Zhuoshi Pan, Yu Li, Honglin Lin +7
Large language models (LLMs) have demonstrated remarkable reasoning capability in solving mathematical problems. However, existing approaches primarily focus on improving the quali…