4 papers · 1 filter
Cluster-Level Attention-Guided Parallel Decoding for Masked Diffusion Language Models
Heqiang Qi, Wei Huang, Mingyuan Bai +1
Masked diffusion language models (MDLMs) enable parallel decoding by predicting all masked positions at each denoising step, yet existing training-free samplers usually decide whic…
LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning
Wei Huang, Anda Cheng, Yinggui Wang +2
Large Language Models (LLMs) can be fine-tuned on domain-specific data to enhance their performance in specialized fields. However, such data often contains numerous low-quality sa…
Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
Wei Huang, Anda Cheng, Yinggui Wang
Recent advancements in large language models (LLMs) have shown impressive capabilities in various downstream tasks but typically face Catastrophic Forgetting (CF) during fine-tunin…
DPF-CM: A Data Processing Framework with Privacy-Preserving Vector Databases for Chinese Medical LLMs Training and Deployment
Wei Huang, Anda Cheng, Zhao Zhang +1
Current open-source training pipelines for Chinese medical language models predominantly emphasize optimizing training methodologies to enhance the performance of large language mo…