2 citations · 4 across the 5 of their papers we have counts for
5 papers
Diagnosing Training Inference Mismatch in LLM Reinforcement Learning via a Zero-Mismatch Reference
Tianle Zhong, Neiwen Ling, Yifan Pi +5
Modern LLM RL systems separate rollout generation from policy optimization. These two stages are expected to produce token probabilities that match exactly. However, implementation…
RINAS: Training with Dataset Shuffling Can Be General and Fast
Tianle Zhong, Jiechen Zhao, Xindi Guo +2
Deep learning datasets are expanding at an unprecedented pace, creating new challenges for data processing in model training pipelines. A crucial aspect of these pipelines is datas…
RTP: Rethinking Tensor Parallelism with Memory Deduplication
Cheng Luo, Tianle Zhong, Geoffrey Fox
In the evolving landscape of neural network models, one prominent challenge stand out: the significant memory overheads associated with training expansive models. Addressing this c…
Supercharging Distributed Computing Environments For High Performance Data Engineering
Niranda Perera, Kaiying Shan, Supun Kamburugamuwe +7
The data engineering and data science community has embraced the idea of using Python & R dataframes for regular applications. Driven by the big data revolution and artificial inte…
Hybrid Cloud and HPC Approach to High-Performance Dataframes
Kaiying Shan, Niranda Perera, Damitha Lenadora +6
Data pre-processing is a fundamental component in any data-driven application. With the increasing complexity of data processing operations and volume of data, Cylon, a distributed…