7 papers
Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models
Xiaoxuan Liao, Chihang Wang, Shicheng Zhou +3
This paper presents a novel methodology of fine-tuning for large language models-dynamic LoRA. Building from the standard Low-Rank Adaptation framework, this methodology further ad…
Feature Alignment-Based Knowledge Distillation for Efficient Compression of Large Language Models
Shuo Wang, Chihang Wang, Jia Gao +3
This study proposes a knowledge distillation algorithm based on large language models and feature alignment, aiming to effectively transfer the knowledge of large pre-trained model…
Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks
Jiacheng Hu, Xiaoxuan Liao, Jia Gao +3
This study proposes a large language model optimization method based on the improved LoRA fine-tuning algorithm, aiming to improve the accuracy and computational efficiency of the…
Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models
Zhen Qi, Jiajing Chen, Shuo Wang +3
This study aims to explore the performance improvement method of large language models based on GPT-4 under the multi-task learning framework and conducts experiments on two tasks:…
A Combined Encoder and Transformer Approach for Coherent and High-Quality Text Generation
Jiajing Chen, Shuo Wang, Zhen Qi +3
This research introduces a novel text generation model that combines BERT's semantic interpretation strengths with GPT-4's generative capabilities, establishing a high standard in…
Advanced RAG Models with Graph Structures: Optimizing Complex Knowledge Reasoning and Text Generation
Yuxin Dong, Shuo Wang, Hongye Zheng +3
This study aims to optimize the existing retrieval-augmented generation model (RAG) by introducing a graph structure to improve the performance of the model in dealing with complex…