4 papers
LLMTailor: A Layer-wise Tailoring Tool for Efficient Checkpointing of Large Language Models
Minqiu Sun, Xin Huang, Luanzheng Guo +3
Checkpointing is essential for fault tolerance in training large language models (LLMs). However, existing methods, regardless of their I/O strategies, periodically store the entir…
Scrutinizing Variables for Checkpoint Using Automatic Differentiation
Xin Huang, Weiping Zhang, Shiman Meng +4
Checkpoint/Restart (C/R) saves the running state of the programs periodically, which consumes considerable system resources. We observe that not every piece of data is involved in…
Distributed Order Recording Techniques for Efficient Record-and-Replay of Multi-threaded Programs
Xiang Fu, Shiman Meng, Weiping Zhang +6
After all these years and all these other shared memory programming frameworks, OpenMP is still the most popular one. However, its greater levels of non-deterministic execution mak…
AutoCheck: Automatically Identifying Variables for Checkpointing by Data Dependency Analysis
Xiang Fu, Weiping Zhang, Xin Huang +4
Checkpoint/Restart (C/R) has been widely deployed in numerous HPC systems, Clouds, and industrial data centers, which are typically operated by system engineers. Nevertheless, ther…