19 citations · 29 across the 9 of their papers we have counts for
9 papers
LayerCheck: Adaptive Layer-wise Checkpointing for Large Language Model Post-training
Minqiu Sun, Xin Huang, Luanzheng Guo +3
With the rising computational and monetary costs of training large language models (LLMs), checkpointing---periodically storing model states for recovery---becomes essential for fa…
ZOCheck: CPU-Shadow Checkpointing for Zeroth-Order LLM Fine-Tuning
Minqiu Sun, Xin Huang, Luanzheng Guo +3
Zeroth-order (ZO) optimization is an attractive option for memory-efficient LLM fine-tuning, but its fault tolerance remains underexplored. Unlike first-order training, ZO progress…
IOAgent: Democratizing Trustworthy HPC I/O Performance Diagnosis Capability via LLMs
Chris Egersdoerfer, Arnav Sareen, Jean Luca Bez +3
As the complexity of the HPC storage stack rapidly grows, domain scientists face increasing challenges in effectively utilizing HPC storage systems to achieve their desired I/O per…
STELLAR: Storage Tuning Engine Leveraging LLM Autonomous Reasoning for High Performance Parallel File Systems
Chris Egersdoerfer, Philip Carns, Shane Snyder +2
I/O performance is crucial to efficiency in data-intensive scientific computing; but tuning large-scale storage systems is complex, costly, and notoriously manpower-intensive, maki…
LLMTailor: A Layer-wise Tailoring Tool for Efficient Checkpointing of Large Language Models
Minqiu Sun, Xin Huang, Luanzheng Guo +3
Checkpointing is essential for fault tolerance in training large language models (LLMs). However, existing methods, regardless of their I/O strategies, periodically store the entir…
Improving SpGEMM Performance Through Matrix Reordering and Cluster-wise Computation
Abdullah Al Raqibul Islam, Helen Xu, Dong Dai +1
Sparse matrix-sparse matrix multiplication (SpGEMM) is a key kernel in many scientific applications and graph workloads. Unfortunately, SpGEMM is bottlenecked by data movement due…
TSUE: A Two-Stage Data Update Method for an Erasure Coded Cluster File System
Zheng Wei, Jing Xing, Yida Gu +4
Compared to replication-based storage systems, erasure-coded storage incurs significantly higher overhead during data updates. To address this issue, various parity logging methods…
A Reinforcement Learning Based Backfilling Strategy for HPC Batch Jobs
Elliot Kolker-Hicks, Di Zhang, Dong Dai
High Performance Computing (HPC) systems are used across a wide range of disciplines for both large and complex computations. HPC systems often receive many thousands of computatio…
DGAP: Efficient Dynamic Graph Analysis on Persistent Memory
Abdullah Al Raqibul Islam, Dong Dai
Dynamic graphs, featuring continuously updated vertices and edges, have grown in importance for numerous real-world applications. To accommodate this, graph frameworks, particularl…