9 papers · 1 filter
Scaling LLM Knowledge Boundaries via Distribution-Optimized Synthesis
Songze Li, Yarong Lan, Zhongpu Bo +16
Knowledge injection via synthetic data is crucial for enhancing Large Language Models (LLMs). However, current synthesis methods simply stop at preset token counts or fixed data ra…
LookAhead Tuning: Safer Language Models via Partial Answer Previews
Kangwei Liu, Mengru Wang, Yujie Luo +7
Fine-tuning enables large language models (LLMs) to adapt to specific domains, but often compromises their previously established safety alignment. To mitigate the degradation of m…
Towards Greater Leverage: Scaling Laws for Efficient Mixture-of-Experts Language Models
Changxin Tian, Kunlong Chen, Jia Liu +3
Mixture-of-Experts (MoE) has become a dominant architecture for scaling Large Language Models (LLMs) efficiently by decoupling total parameters from computational cost. However, th…
MAQInstruct: Instruction-based Unified Event Relation Extraction
Jun Xu, Mengshu Sun, Zhiqiang Zhang +1
Extracting event relations that deviate from known schemas has proven challenging for previous methods based on multi-class classification, MASK prediction, or prototype matching.…
Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis
Lin Yuan, Jun Xu, Honghao Gui +4
High-quality, large-scale instructions are crucial for aligning large language models (LLMs), however, there is a severe shortage of instruction in the field of natural language un…
OneKE: A Dockerized Schema-Guided LLM Agent-based Knowledge Extraction System
Yujie Luo, Xiangyuan Ru, Kangwei Liu +10
We introduce OneKE, a dockerized schema-guided knowledge extraction system, which can extract knowledge from the Web and raw PDF Books, and support various domains (science, news,…