From the 1 of 17 linked papers with an AI index.
6 papers · 1 filter
MemTrain: Self-Supervised Context Memory Training
Ziheng Li, Xingrun Xing, Haoqing Wang +2
Memory is an indispensable capability for long-horizon LLM agents, enabling them to preserve and utilize information accumulated across extended interactions. Existing memory-agent…
PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference
Hao Zhang, Mengsi Lyu, Zhuo Chen +3
Large Language Models (LLMs) demonstrate exceptional capabilities across various tasks, but their deployment is constrained by high computational and memory costs. Model pruning pr…
PretrainZero: Reinforcement Active Pretraining
Xingrun Xing, Zhiyuan Fan, Jie Lou +3
Mimicking human behavior to actively learning from general experience and achieve artificial general intelligence has always been a human dream. Recent reinforcement learning (RL)…
Position-Aware Depth Decay Decoding (): Boosting Large Language Model Inference Efficiency
Siqi Fan, Xuezhi Fang, Xingrun Xing +3
Due to the large number of parameters, the inference phase of Large Language Models (LLMs) is resource-intensive. Unlike traditional model compression, which needs retraining, rece…
MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts
Shujun Xia, Haokun Lin, Yichen Wu +9
LLMs hold great promise for healthcare applications, but the rapid evolution of medical knowledge and errors in training data often cause them to generate outdated or inaccurate in…
Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
Yiming Ju, Ziyi Ni, Xingrun Xing +4
Supervised fine-tuning (SFT) is crucial for adapting Large Language Models (LLMs) to specific tasks. In this work, we demonstrate that the order of training data can lead to signif…