4 papers
NeuronMLP: Efficient LLM Inference via Singular Value Decomposition Compression and Tiling on AWS Trainium
Dinghong Song, Jierui Xu, Weichu Yang +2
Emerging AI accelerators have started to gain attention and offer new opportunities for efficient inference of large language models (LLMs). Trainium, an AI accelerator recently de…
CoCoDiff: Optimizing Collective Communications for Distributed Diffusion Transformer Inference Under Ulysses Sequence Parallelism
Bin Ma, Xingjian Ding, Tekin Bicer +2
Diffusion Transformers (DiTs) are increasingly adopted in scientific computing, yet growing model sizes and resolutions make distributed multi-GPU inference essential. Ulysses sequ…
Adversarial Contrastive Learning for LLM Quantization Attacks
Dinghong Song, Zhiwei Xu, Hai Wan +3
Model quantization is critical for deploying large language models (LLMs) on resource-constrained hardware, yet recent work has revealed severe security risks that benign LLMs in f…
AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache
Dinghong Song, Yuan Feng, Yiwei Wang +6
Large Language Models (LLMs) are widely used in generative applications such as chatting, code generation, and reasoning. However, many realworld workloads such as classification,…