From the 1 of 12 linked papers with an AI index.
12 papers
Beyond KV Reconstruction: Functional Reconstruction for MLA Draft Models in Speculative Decoding
Weiye Shi, Fanxu Meng, Muhan Zhang
The paper proposes a functional reconstruction method to convert multi-head or grouped-query attention models into multi-head latent attention form, preserving cache efficiency and…
LiteToken: Removing Intermediate Merge Residues From BPE Tokenizers
Yike Sun, Haotong Yang, Zhouchen Lin +1
Tokenization is fundamental to how language models represent and process text, yet the behavior of widely used BPE tokenizers has received far less study than model architectures a…
Breaking the Blocks: Continuous Low-Rank Decomposed Scaling for Unified LLM Quantization and Adaptation
Pingzhi Tang, Ruijie Zhou, Fanxu Meng +2
Current quantization methods for LLMs predominantly rely on block-wise structures to maintain efficiency, often at the cost of representational flexibility. In this work, we demons…
HD-PiSSA: High-Rank Distributed Orthogonal Adaptation
Yiding Wang, Fauxu Meng, Xuefeng Zhang +3
Existing parameter-efficient fine-tuning (PEFT) methods for large language models (LLMs), such as LoRA and PiSSA, constrain model updates to low-rank subspaces, limiting their expr…
TransMLA: Multi-Head Latent Attention Is All You Need
Fanxu Meng, Pingzhi Tang, Xiaojuan Tang +3
In this paper, we present TransMLA, a framework that seamlessly converts any GQA-based pre-trained model into an MLA-based model. Our approach enables direct compatibility with Dee…
Beyond Single-Task: Robust Multi-Task Length Generalization for LLMs
Yi Hu, Shijia Kang, Haotong Yang +2
Length generalization, the ability to solve problems longer than those seen during training, remains a critical challenge for large language models (LLMs). Previous work modifies p…