6 papers
Towards Stepwise Domain Knowledge-Driven Reasoning Optimization and Reflection Improvement
Chengyuan Liu, Shihang Wang, Lizhi Qing +7
Recently, stepwise supervision on Chain of Thoughts (CoTs) presents an enhancement on the logical reasoning tasks such as coding and math, with the help of Monte Carlo Tree Search…
Personalized LLM Response Generation with Parameterized Memory Injection
Kai Zhang, Yejin Kim, Xiaozhong Liu
Large Language Models (LLMs) have exhibited remarkable proficiency in comprehending and generating natural language. On the other hand, personalized LLM response generation holds t…
Learning to Solve Domain-Specific Calculation Problems with Knowledge-Intensive Programs Generator
Chengyuan Liu, Shihang Wang, Lizhi Qing +4
Domain Large Language Models (LLMs) are developed for domain-specific tasks based on general LLMs. But it still requires professional knowledge to facilitate the expertise for some…
Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMs
Chengyuan Liu, Shihang Wang, Lizhi Qing +4
While Large Language Models (LLMs) demonstrate impressive generation abilities, they frequently struggle when it comes to specialized domains due to their limited domain-specific k…
More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMs
Chengyuan Liu, Yangyang Kang, Shihang Wang +5
The performance on general tasks decreases after Large Language Models (LLMs) are fine-tuned on domain-specific tasks, the phenomenon is known as Catastrophic Forgetting (CF). Howe…
Enhance Robustness of Language Models Against Variation Attack through Graph Integration
Zi Xiong, Lizhi Qing, Yangyang Kang +5
The widespread use of pre-trained language models (PLMs) in natural language processing (NLP) has greatly improved performance outcomes. However, these models' vulnerability to adv…