74 citations · 219 across the 12 of their papers we have counts for
13 papers
Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
Mert Hidayetoglu, Aurick Qiao, Michael Wyatt +3
Efficient parallelism is necessary for achieving low-latency, high-throughput inference with large language models (LLMs). Tensor parallelism (TP) is the state-of-the-art method fo…
Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI
Samyam Rajbhandari, Mert Hidayetoglu, Aurick Qiao +5
Inference is now the dominant AI workload, yet existing systems force trade-offs between latency, throughput, and cost. Arctic Inference, an open-source vLLM plugin from Snowflake…
Federated Timeline Synthesis: Scalable and Private Methodology For Model Training and Deployment
Pawel Renc, Michal K. Grzeszczyk, Linglong Qian +3
We present Federated Timeline Synthesis (FTS), a novel framework for training generative foundation models across distributed timeseries data applied to electronic health records (…
Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences
Stas Bekman, Samyam Rajbhandari, Michael Wyatt +5
Long sequences are critical for applications like RAG, long document summarization, multi-modality, etc., and modern LLMs, like Llama 4 Scout, support max sequence length of up to…
DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
Connor Holmes, Masahiro Tanaka, Michael Wyatt +8
The deployment and scaling of large language models (LLMs) have become critical as they permeate various applications, demanding high-throughput and low-latency serving systems. Ex…
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang +89
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scient…