5 papers
Gradient Under Microscope: Benchmarking Resource Utilization of Memory-Efficient Gradient Computation Methods
Sarthak Mahapatra, Zihan Zhou, Khatoon Khedri +2
AI training's rising resource intensity is straining electricity supplies and carbon budgets, motivating systematic study of memory-efficient training on constrained hardware. We b…
Transcriptomic Models for Immunotherapy Response Prediction Show Limited Cross-cohort Generalisability
Yuheng Liang, Lucy Chhuo, Ahmadreza Argha +8
Immune checkpoint inhibitors (ICIs) have transformed cancer therapy; yet substantial proportion of patients exhibit intrinsic or acquired resistance, making accurate pre-treatment…
Pruning and Quantization Impact on Graph Neural Networks
Khatoon Khedri, Reza Rawassizadeh, Qifu Wen +1
Graph neural networks (GNNs) are known to operate with high accuracy on learning from graph-structured data, but they suffer from high computational and resource costs. Neural netw…
GradES: Significantly Faster Training in Transformers with Gradient-Based Early Stopping
Qifu Wen, Xi Zeng, Zihan Zhou +4
Early stopping monitors global validation loss and halts all parameter updates simultaneously, which is computationally costly for large transformers due to the extended time requi…
TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization
Hainan Wang, Mehdi Hosseinzadeh, Reza Rawassizadeh
The success of the generative model has gained unprecedented attention in the music generation area. Transformer-based architectures have set new benchmarks for model performance.…