4 papers · 1 filter
Pier: Efficient Large Language Model pretraining with Relaxed Global Communication
Shuyuan Fan, Zhao Zhang
Global communication, such as all-reduce and allgather, is the prominent performance bottleneck in large language model (LLM) pretraining. To address this issue, we present Pier, a…
When Harmless Words Harm: A New Threat to LLM Safety via Conceptual Triggers
Zhaoxin Zhang, Borui Chen, Yiming Hu +3
Recent research on large language model (LLM) jailbreaks has primarily focused on techniques that bypass safety mechanisms to elicit overtly harmful outputs. However, such efforts…
Maple: A Multi-agent System for Portable Deep Learning across Clusters
Molang Wu, Zhao Zhang
Training deep learning (DL) models across Graphics Processing Unit (GPU) clusters is technically challenging. One aspect is that users have to compose command lines to adapt to the…
Efficient Fine-Grained GPU Performance Modeling for Distributed Deep Learning of LLM
Biyao Zhang, Mingkai Zheng, Debargha Ganguly +4
Training Large Language Models(LLMs) is one of the most compute-intensive tasks in high-performance computing. Predicting end-to-end training time for multi-billion parameter model…