14 papers
Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo +4
Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation, leveraging parallel decoding and bidirectional attention to achieve superior gl…
Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks
Wenbo Pan, Jie Xu, Qiguang Chen +5
Large Language Models (LLMs) should refuse to answer questions beyond their knowledge. This capability, which we term knowledge-aware refusal, is crucial for factual reliability, w…
Dynamic Noise Preference Optimization: Self-Improvement of Large Language Models with Self-Synthetic Data
Haoyan Yang, Khiem Le, Ting Hua +7
Although LLMs have achieved significant success, their reliance on large volumes of human-annotated data has limited their potential for further scaling. In this situation, utilizi…
Contamination Means Overestimation? A Fine-Grained Empirical Study in Code Intelligence
Zhen Yang, Hongyi Lin, Yifan He +7
In recent years, code intelligence has gained increasing importance in the field of automated software engineering. Meanwhile, the widespread adoption of Pretrained Language Models…
Noise-Adaptive Layerwise Learning Rates: Accelerating Geometry-Aware Optimization for Deep Neural Network Training
Jie Hao, Xiaochuan Gong, Jie Xu +2
Geometry-aware optimization algorithms, such as Muon, have achieved remarkable success in training deep neural networks (DNNs). These methods leverage the underlying geometry of DN…
Farewell to Item IDs: Unlocking the Scaling Potential of Large Ranking Models via Semantic Tokens
Zhen Zhao, Tong Zhang, Jie Xu +5
Recent studies on scaling up ranking models have achieved substantial improvement for recommendation systems and search engines. However, most large-scale ranking systems rely on i…