1 citations · 1 across the 4 of their papers we have counts for
4 papers
SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile
Ruisi Zhang, Tianyu Liu, Will Feng +4
Distributed training of large models consumes enormous computation resources and requires substantial engineering efforts to compose various training techniques. This paper present…
Flash Inference: Near Linear Time Inference for Long Convolution Sequence Models and Beyond
Costin-Andrei Oncescu, Sanket Purandare, Stratos Idreos +1
While transformers have been at the core of most recent advancements in sequence generative models, their computational cost remains quadratic in sequence length. Several subquadra…
TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training
Wanchao Liang, Tianyu Liu, Less Wright +10
The development of large language models (LLMs) has been instrumental in advancing state-of-the-art natural language processing applications. Training LLMs with billions of paramet…
Optimizing Cursor Loops in Relational Databases
Surabhi Gupta, Sanket Purandare, Karthik Ramachandra
Loops that iterate over SQL query results are quite common, both in application programs that run outside the DBMS, as well as User Defined Functions (UDFs) and stored procedures t…