3 papers
cs.LG2026
ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers
Mahdi Heidari, Mohammad Mahdi Rahimi, Jaekyun Moon
The quadratic attention score matrix remains a central obstacle to extending Transformers to longer input lengths. Existing efficient attention methods usually reduce t…
cs.LG2026
PruneFuse: Efficient Data Selection via Weight Pruning and Network Fusion
Humaira Kousar, Hasnain Irshad Bhatti, Jaekyun Moon
Efficient data selection is crucial for enhancing the training efficiency of deep neural networks and minimizing annotation requirements. Traditional methods often face high comput…
cs.LG2025
Pruning-based Data Selection and Network Fusion for Efficient Deep Learning
Humaira Kousar, Hasnain Irshad Bhatti, Jaekyun Moon
Efficient data selection is essential for improving the training efficiency of deep neural networks and reducing the associated annotation costs. However, traditional methods tend…