10 papers
PDFBench: A Benchmark for De novo Protein Design from Function
Jiahao Kuang, Nuowei Liu, Jie Wang +3
Function-guided protein design is a crucial task with significant applications in drug discovery and enzyme engineering. However, the field lacks a unified and comprehensive evalua…
TASO: Task-Aligned Sparse Optimization for Parameter-Efficient Model Adaptation
Daiye Miao, Yufang Liu, Jie Wang +6
LoRA has become one of the most widely used parameter-efficient fine-tuning methods due to its simplicity and effectiveness. However, numerous studies have shown that LoRA often in…
DVAGen: Dynamic Vocabulary Augmented Generation
Wei Du, Nuowei Liu, Jie Wang +4
Language models trained with a fixed vocabulary struggle to generalize to novel or out-of-vocabulary words, limiting their flexibility in handling diverse token combinations. Exist…
Protein Design with Dynamic Protein Vocabulary
Nuowei Liu, Jiahao Kuang, Yanting Liu +4
Protein design is a fundamental challenge in biotechnology, aiming to design novel sequences with specific functions within the vast space of possible proteins. Recent advances in…
Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
Tao Ji, Bin Guo, Yuanbin Wu +6
Multi-head Latent Attention (MLA) is an innovative architecture proposed by DeepSeek, designed to ensure efficient and economical inference by significantly compressing the Key-Val…
On Support Samples of Next Word Prediction
Yuqian Li, Yupei Du, Yufang Liu +3
Language models excel in various tasks by making complex decisions, yet understanding the rationale behind these decisions remains a challenge. This paper investigates \emph{data-c…