7 papers
Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts
Zixuan Huang, Da Chen, Kecheng Huang +5
Generating high-performance GPU kernels remains challenging due to the need for both correctness and hardware-aware optimization. While large language models (LLMs) show promise in…
What Matters For Safety Alignment?
Xing Li, Hui-Ling Zhen, Lihao Yin +3
This paper presents a comprehensive empirical study on the safety alignment capabilities. We evaluate what matters for safety alignment in LLMs and LRMs to provide essential insigh…
PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery
Bowei He, Lihao Yin, Hui-Ling Zhen +3
Model pruning is an effective approach for compressing large language models (LLMs). However, this process often leads to significant degradation of model capabilities. While post-…
DiLA: Enhancing LLM Tool Learning with Differential Logic Layer
Yu Zhang, Hui-Ling Zhen, Zehua Pei +4
Considering the challenges faced by large language models (LLMs) in logical reasoning and planning, prior efforts have sought to augment LLMs with access to external solvers. While…
Attention-Aware GNN-based Input Defense against Multi-Turn LLM Jailbreak
Zixuan Huang, Kecheng Huang, Lihao Yin +4
Large Language Models (LLMs) have gained significant traction in various applications, yet their capabilities present risks for both constructive and malicious exploitation. Despit…
Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
Bowei He, Lihao Yin, Huiling Zhen +5
Post-training compression has been a widely employed approach to scale down large language model (LLM) and facilitate efficient inference. In various proposed compression methods,…