26 citations · 28 across the 12 of their papers we have counts for
13 papers
Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalization
Qian Kou, Xiaofeng Shi, Xiaosong Qiu +1
Large language models often fail to answer questions about a bounded document collection when the source documents are not retrieved at inference time. We study this setting as doc…
Wnuan: Staged Post-Training for Question Answering over Proprietary Enterprise Knowledge
Xiaofeng Shi, Xiaosong Qiu, Wenxin Ma +6
Enterprise question answering requires models to acquire proprietary knowledge without discarding general capabilities. We present Wnuan, a three-stage pipeline that constructs tas…
RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
Yuduo Li, Xiaofeng Shi, Qian Kou +2
Domain-specific supervised fine-tuning (SFT) often improves in-domain performance at the cost of degrading a model's general capabilities. We view this degradation through two prac…
Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy
Xiaofeng Shi, Qian Kou, Yuduo Li +1
With the rapid advancement of Large Language Models (LLMs), the Chain-of-Thought (CoT) component has become significant for complex reasoning tasks. However, in conventional Superv…
Design, Results and Industry Implications of the World's First Insurance Large Language Model Evaluation Benchmark
Hua Zhou, Bing Ma, Yufei Zhang +1
This paper comprehensively elaborates on the construction methodology, multi-dimensional evaluation system, and underlying design philosophy of CUFEInse v1.0. Adhering to the princ…
SPAR: Scholar Paper Retrieval with LLM-based Agents for Enhanced Academic Search
Xiaofeng Shi, Yuduo Li, Qian Kou +3
Recent advances in large language models (LLMs) have opened new opportunities for academic literature retrieval. However, existing systems often rely on rigid pipelines and exhibit…
SciSage: A Multi-Agent Framework for High-Quality Scientific Survey Generation
Xiaofeng Shi, Qian Kou, Yuduo Li +5
The rapid growth of scientific literature demands robust tools for automated survey-generation. However, current large language model (LLM)-based methods often lack in-depth analys…
CareBot: A Pioneering Full-Process Open-Source Medical Language Model
Lulu Zhao, Weihao Zeng, Xiaofeng Shi +1
Recently, both closed-source LLMs and open-source communities have made significant strides, outperforming humans in various general domains. However, their performance in specific…
Smaller Language Models Are Better Instruction Evolvers
Tingfeng Hui, Lulu Zhao, Guanting Dong +3
Instruction tuning has been widely used to unleash the complete potential of large language models. Notably, complex and diverse instructions are of significant importance as they…
MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning
Lulu Zhao, Weihao Zeng, Xiaofeng Shi +1
Recently, LoRA has emerged as a crucial technique for fine-tuning large pre-trained models, yet its performance in multi-task learning scenarios often falls short. In contrast, the…
Robust Noisy Correspondence Learning via Self-Drop and Dual-Weight
Fan Liu, Chenwei Dong, Chuanyi Zhang +2
Many researchers collect data from the internet through crowd-sourcing or web crawling to alleviate the data-hungry challenge associated with cross-modal matching. Although such pr…
MiniCPM-V: A GPT-4V Level MLLM on Your Phone
Yuan Yao, Tianyu Yu, Ao Zhang +20
The recent surge of Multimodal Large Language Models (MLLMs) has fundamentally reshaped the landscape of AI research and industry, shedding light on a promising path toward the nex…
Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Models
Lulu Zhao, Weihao Zeng, Xiaofeng Shi +3
Recently, both closed-source LLMs and open-source communities have made significant strides, outperforming humans in various general domains. However, their performance in specific…