3 papers
cs.DC2026
Accuracy Is Speed: Towards Long-Context-Aware Routing for Distributed LLM Serving
Takeshi Yoshimura, Valentijn Dymphnus van de Beek, Tatsuhiro Chiba
Distributed LLM serving systems optimize per-request latency and throughput. However, under long-context workloads, inference accuracy becomes more variable. When incorrect respons…
cs.DC2025
Speeding up Model Loading with fastsafetensors
Takeshi Yoshimura, Tatsuhiro Chiba, Manish Sethi +2
The rapid increases in model parameter sizes introduces new challenges in pre-trained model loading. Currently, machine learning code often deserializes each parameter as a tensor…
cs.DC2025
The infrastructure powering IBM's Gen AI model development
Talia Gershon, Seetharami Seelam, Brian Belgodere +143
AI Infrastructure plays a key role in the speed and cost-competitiveness of developing and deploying advanced AI models. The current demand for powerful AI infrastructure for model…