6 citations · 6 across the 11 of their papers we have counts for
14 papers
A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMs
Jiahao Kuang, Yangfan Gao, Tao Gui +4
Parallel generation is widely regarded as an inherent advantage of diffusion large language models. Yet our fixed-quota evaluation of four representative models reveals severe degr…
Byte Language Models: Scaling, Emergent Abstractions, and Information Allocation
Jie Wang, Shiwei Luo, Qi Zhang +1
Tokenizer-free language models remove the inductive bias of fixed tokenizers by modeling text directly as bytes, but the resulting longer sequences substantially increase computati…
DVAGen: Dynamic Vocabulary Augmented Generation
Wei Du, Nuowei Liu, Jie Wang +4
Language models trained with a fixed vocabulary struggle to generalize to novel or out-of-vocabulary words, limiting their flexibility in handling diverse token combinations. Exist…
TASO: Task-Aligned Sparse Optimization for Parameter-Efficient Model Adaptation
Daiye Miao, Yufang Liu, Jie Wang +6
LoRA has become one of the most widely used parameter-efficient fine-tuning methods due to its simplicity and effectiveness. However, numerous studies have shown that LoRA often in…
On Support Samples of Next Word Prediction
Yuqian Li, Yupei Du, Yufang Liu +3
Language models excel in various tasks by making complex decisions, yet understanding the rationale behind these decisions remains a challenge. This paper investigates \emph{data-c…
PDFBench: A Benchmark for De novo Protein Design from Function
Jiahao Kuang, Nuowei Liu, Jie Wang +3
Function-guided protein design is a crucial task with significant applications in drug discovery and enzyme engineering. However, the field lacks a unified and comprehensive evalua…
Protein Design with Dynamic Protein Vocabulary
Nuowei Liu, Jiahao Kuang, Yanting Liu +4
Protein design is a fundamental challenge in biotechnology, aiming to design novel sequences with specific functions within the vast space of possible proteins. Recent advances in…
The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights
Yufang Liu, Yao Du, Tao Ji +6
Recent research has increasingly focused on multimodal mathematical reasoning, particularly emphasizing the creation of relevant datasets and benchmarks. Despite this, the role of…
Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
Tao Ji, Bin Guo, Yuanbin Wu +6
Multi-head Latent Attention (MLA) is an innovative architecture proposed by DeepSeek, designed to ensure efficient and economical inference by significantly compressing the Key-Val…
AntLM: Bridging Causal and Masked Language Models
Xinru Yu, Bin Guo, Shiwei Luo +3
Causal Language Modeling (CLM) and Masked Language Modeling (MLM) are two mainstream learning paradigms based on Transformer networks, specifically the Decoder-only and Encoder-onl…
Generation with Dynamic Vocabulary
Yanting Liu, Tao Ji, Changzhi Sun +2
We introduce a new dynamic vocabulary for language models. It can involve arbitrary text spans during generation. These text spans act as basic generation bricks, akin to tokens in…
Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models
Yufang Liu, Tao Ji, Changzhi Sun +2
Large Vision-Language Models (LVLMs) have achieved impressive performance, yet research has pointed out a serious issue with object hallucinations within these models. However, the…
A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications
Pengfei Wang, Huanran Zheng, Silong Dai +4
In recent years, the field of electroencephalography (EEG) analysis has witnessed remarkable advancements, driven by the integration of machine learning and artificial intelligence…
Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis
Xuanwen Ding, Jie Zhou, Liang Dou +4
Aspect-based sentiment analysis (ABSA) is an important subtask of sentiment analysis, which aims to extract the aspects and predict their sentiments. Most existing studies focus on…