5 papers
RAC: Reference-Aware Activation Compression for Communication-Efficient Split LLM Inference
Guotao Yang, Mingxi Zhao, Haopeng Li +4
Large language model (LLM) agents repeatedly process long, privacy-sensitive contexts, while cloud-only deployment exposes user data beyond the trusted endpoint and fully local dep…
KernelSkill: A Multi-Agent Framework for GPU Kernel Optimization
Qitong Sun, Jun Han, Tianlin Li +6
Improving GPU kernel efficiency is crucial for advancing AI systems. Recent work has explored leveraging large language models (LLMs) for GPU kernel generation and optimization. Ho…
Mosaic: Unlocking Long-Context Inference for Diffusion LLMs via Global Memory Planning and Dynamic Peak Taming
Liang Zheng, Bowen Shi, Yitao Hu +5
Diffusion-based large language models (dLLMs) have emerged as a promising paradigm, utilizing simultaneous denoising to enable global planning and iterative refinement. While these…
dots.llm1 Technical Report
Bi Huo, Bin Tu, Cheng Qin +24
Mixture of Experts (MoE) models have emerged as a promising paradigm for scaling language models efficiently by activating only a subset of parameters for each input token. In this…
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies
Luping Wang, Sheng Chen, Linnan Jiang +4
The large models, as predicted by scaling raw forecasts, have made groundbreaking progress in many fields, particularly in natural language generation tasks, where they have approa…