Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
Tao Ji, Bin Guo, Yuanbin Wu +6
Multi-head Latent Attention (MLA) is an innovative architecture proposed by DeepSeek, designed to ensure efficient and economical inference by significantly compressing the Key-Val…
cs.CL2024
AntLM: Bridging Causal and Masked Language Models
Xinru Yu, Bin Guo, Shiwei Luo +3
Causal Language Modeling (CLM) and Masked Language Modeling (MLM) are two mainstream learning paradigms based on Transformer networks, specifically the Decoder-only and Encoder-onl…