11 papers
Normalized Low-Rank Adaptation
Jiale Kang, Ziyin Yue, Zheng Zhan +2
While low-rank adaptation (LoRA) is widely used for parameter-efficient model adaptation, how to regularize its training dynamics for stable and effective optimization remains unde…
XYZFlow:Scaling Multi dimensional Shortcut Flows for Efficient Generative Modeling
Jinxiu Liu, Xuanming Liu, Kangfu Mei +2
High-fidelity image generation faces a trade-off between speed and quality. Diffusion models produce strong visuals but require costly iterative sampling. Existing efficient method…
SymbOmni: Evolving Agentic Omni Models via Symbolic Concept Learning
Jinxiu Liu, Jianru Li, Tanqing Kuang +4
Visual generation is increasingly ubiquitous in diverse domains, from text-to-image/video synthesis to multimodal interactive creation. Yet prevailing monolithic models remain fund…
Sample-Efficient Post-Training for LEGO Spatial-Physics Reasoning
Yuhuan Yuan, Zhouliang Yu, Minghao Liu +2
LLM-based LEGO assembly requires both semantic grounding and physical feasibility. In this paper, we identify a data-induced failure mode, physhack, in which generated assemblies s…
PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
Yangyi Huang, Ruotian Peng, Zeju Qiu +4
Parameter-efficient finetuning (PEFT) has become the standard approach for adapting large language models, yet evaluations largely emphasize downstream accuracy while overlooking t…
Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
Kexuan Shi, Hanxuan Li, Zeju Qiu +3
We introduce Pion, a spectrum-preserving optimizer for large language model (LLM) training based on orthogonal equivalence transformation. Unlike additive optimizers such as Adam a…
Streaming Autoregressive Video Generation via Diagonal Distillation
Jinxiu Liu, Xuanming Liu, Kangfu Mei +3
Large pretrained diffusion models have significantly enhanced the quality of generated videos, and yet their use in real-time streaming remains limited. Autoregressive models offer…
Model Merging with Functional Dual Anchors
Kexuan Shi, Yandong Wen, Weiyang Liu
Model merging is an efficient post-training strategy for integrating knowledge from multiple finetuned checkpoints of a shared foundation model. Existing methods operate in the par…
Beyond the Sampled Token: Preserving Candidate Support in RLVR
Ruotian Peng, Yi Ren, Zhouliang Yu +2
We revisit exploration collapse in reinforcement learning with verifiable rewards (RLVR), from the perspective of the \emph{candidate distribution} for next-token prediction. We fo…
Symbolic Graphics Programming with Large Language Models
Yamei Chen, Haoquan Zhang, Yangyi Huang +4
Large language models (LLMs) excel at program synthesis, yet their ability to produce symbolic graphics programs (SGPs) that render into precise visual content remains underexplore…
FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Models
Zhouliang Yu, Ruotian Peng, Keyi Ding +10
Formal mathematical reasoning remains a critical challenge for artificial intelligence, hindered by limitations of existing benchmarks in scope and scale. To address this, we prese…