3 papers
cs.DC2025
AnchorTP: Resilient LLM Inference with State-Preserving Elastic Tensor Parallelism
Wendong Xu, Chujie Chen, He Xiao +8
Large Language Model (LLM) inference services demand exceptionally high availability and low latency, yet multi-GPU Tensor Parallelism (TP) makes them vulnerable to single-GPU fail…
cs.LG2025
PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
He Xiao, Runming Yang, Qingyao Yang +6
Post-training quantization (PTQ) of large language models (LLMs) to extremely low bit-widths remains challenging due to the fundamental trade-off between computational efficiency a…
cs.LG2025
Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Models
He Xiao, Qingyao Yang, Dirui Xie +7
Large language models with billions of parameters are often over-provisioned: many layers contribute little unique information yet dominate the memory and energy footprint during i…