5 papers
P Law: Scaling Law for Post-Training After Model Pruning
Xiaodong Chen, Yuxuan Hu, Xiaokang Zhang +4
Pruning has become a widely adopted technique for reducing the hardware requirements of large language models (LLMs). To recover model performance after pruning, post-training is c…
A Learn-Then-Reason Model Towards Generalization in Knowledge Base Question Answering
Lingxi Zhang, Jing Zhang, Yanling Wang +2
Large-scale knowledge bases (KBs) like Freebase and Wikidata house millions of structured knowledge. Knowledge Base Question Answering (KBQA) provides a user-friendly way to access…
SGSH: Stimulate Large Language Models with Skeleton Heuristics for Knowledge Base Question Generation
Shasha Guo, Lizi Liao, Jing Zhang +3
Knowledge base question generation (KBQG) aims to generate natural language questions from a set of triplet facts extracted from KB. Existing methods have significantly boosted the…
Open-World Semi-Supervised Learning for Node Classification
Yanling Wang, Jing Zhang, Lingxi Zhang +5
Open-world semi-supervised learning (Open-world SSL) for node classification, that classifies unlabeled nodes into seen classes or multiple novel classes, is a practical but under-…
Streamlining Redundant Layers to Compress Large Language Models
Xiaodong Chen, Yuxuan Hu, Jing Zhang +3
This paper introduces LLM-Streamline, a pioneer work on layer pruning for large language models (LLMs). It is based on the observation that different layers have varying impacts on…