4 citations · 14 across the 19 of their papers we have counts for
8 papers · 1 filter
MemTrain: Self-Supervised Context Memory Training
Ziheng Li, Xingrun Xing, Haoqing Wang +2
Memory is an indispensable capability for long-horizon LLM agents, enabling them to preserve and utilize information accumulated across extended interactions. Existing memory-agent…
PretrainZero: Reinforcement Active Pretraining
Xingrun Xing, Zhiyuan Fan, Jie Lou +3
Mimicking human behavior to actively learning from general experience and achieve artificial general intelligence has always been a human dream. Recent reinforcement learning (RL)…
MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts
Shujun Xia, Haokun Lin, Yichen Wu +9
LLMs hold great promise for healthcare applications, but the rapid evolution of medical knowledge and errors in training data often cause them to generate outdated or inaccurate in…
PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference
Hao Zhang, Mengsi Lyu, Zhuo Chen +3
Large Language Models (LLMs) demonstrate exceptional capabilities across various tasks, but their deployment is constrained by high computational and memory costs. Model pruning pr…
Position-Aware Depth Decay Decoding (): Boosting Large Language Model Inference Efficiency
Siqi Fan, Xuezhi Fang, Xingrun Xing +3
Due to the large number of parameters, the inference phase of Large Language Models (LLMs) is resource-intensive. Unlike traditional model compression, which needs retraining, rece…
Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
Yiming Ju, Ziyi Ni, Xingrun Xing +4
Supervised fine-tuning (SFT) is crucial for adapting Large Language Models (LLMs) to specific tasks. In this work, we demonstrate that the order of training data can lead to signif…
LM-Cocktail: Resilient Tuning of Language Models via Model Merging
Shitao Xiao, Zheng Liu, Peitian Zhang +1
The pre-trained language models are continually fine-tuned to better support downstream applications. However, this operation may result in significant performance degeneration on…
KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Models
Yiming Ju, Xingrun Xing, Zhixiong Zeng
Recently, Locate-Then-Edit paradigm has emerged as one of the main approaches in changing factual knowledge stored in the Language models. However, there is a lack of research on w…