1 citations · 1 across the 4 of their papers we have counts for
4 papers · 1 filter
Accuracy Is Speed: Towards Long-Context-Aware Routing for Distributed LLM Serving
Takeshi Yoshimura, Valentijn Dymphnus van de Beek, Tatsuhiro Chiba
Distributed LLM serving systems optimize per-request latency and throughput. However, under long-context workloads, inference accuracy becomes more variable. When incorrect respons…
Speeding up Model Loading with fastsafetensors
Takeshi Yoshimura, Tatsuhiro Chiba, Manish Sethi +2
The rapid increases in model parameter sizes introduces new challenges in pre-trained model loading. Currently, machine learning code often deserializes each parameter as a tensor…
The infrastructure powering IBM's Gen AI model development
Talia Gershon, Seetharami Seelam, Brian Belgodere +143
AI Infrastructure plays a key role in the speed and cost-competitiveness of developing and deploying advanced AI models. The current demand for powerful AI infrastructure for model…
Objcache: An Elastic Filesystem over External Persistent Storage for Container Clusters
Takeshi Yoshimura, Tatsuhiro Chiba, Sunyanan Choochotkaew +3
Container virtualization enables emerging AI workloads such as model serving, highly parallelized training, machine learning pipelines, and so on, to be easily scaled on demand on…