5 papers
BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
Xiaoyou Wu, Cheng-Jhih Shih, Binfei Ji +2
Diffusion language models (dLLMs) generate text by iteratively denoising multiple token positions in parallel, offering an attractive alternative to strictly autoregressive decodin…
AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security
Dongrui Liu, Yu Li, Zhonghao Yang +47
Modern open-world agents such as OpenClaw exhibit powerful cross-environment execution capabilities yet introduce broad new safety risk sources. Meanwhile, advanced frontier AI mod…
Nora: Normalized Orthogonal Row Alignment for Scalable Matrix Optimizer
Jinghui Yuan, Jiaxuan Zou, Shuo Wang +2
Matrix-based optimizers have demonstrated immense potential in training Large Language Models (LLMs), however, designing an ideal optimizer remains a formidable challenge. A superi…
Beyond the Black Box: A Survey on the Theory and Mechanism of Large Language Models
Zeyu Gan, Ruifeng Ren, Wei Yao +9
The rapid emergence of Large Language Models (LLMs) has precipitated a profound paradigm shift in Artificial Intelligence, delivering monumental engineering successes that increasi…
Put the Space of LoRA Initialization to the Extreme to Preserve Pre-trained Knowledge
Pengwei Tang, Xiaolin Hu, Yong Liu +4
Low-Rank Adaptation (LoRA) is the leading parameter-efficient fine-tuning method for Large Language Models (LLMs), but it still suffers from catastrophic forgetting. Recent work ha…