7 papers · 1 filter
Market-Driven Subset Selection for Budgeted Training
Ashish Jha, Valentin Leplat, AH Phan
Training large language models on massive datasets is computationally expensive, yet empirical evidence suggests that substantial portions of training examples contribute minimally…
GRAFT: Gradient-Aware Fast MaxVol Technique for Dynamic Data Sampling
Ashish Jha, Anh huy Phan, Razan Dibo +1
Training modern neural networks on large datasets is computationally and environmentally costly. We introduce GRAFT, a scalable in-training subset selection method that (i) extract…
Ruppert-Polyak averaging for Stochastic Order Oracle
V. N. Smirnov, K. M. Kazistova, I. A. Sudakov +3
Black-box optimization, a rapidly growing field, faces challenges due to limited knowledge of the objective function's internal mechanisms. One promising approach to address this i…
Efficient Algorithms for Regularized Nonnegative Scale-invariant Low-rank Approximation Models
Jeremy E. Cohen, Valentin Leplat
Regularized nonnegative low-rank approximations, such as sparse Nonnegative Matrix Factorization or sparse Nonnegative Tucker Decomposition, form an important branch of dimensional…
Block Majorization Minimization with Extrapolation and Application to -NMF
Le Thi Khanh Hien, Valentin Leplat, Nicolas Gillis
We propose a Block Majorization Minimization method with Extrapolation (BMMe) for solving a class of multi-convex optimization problems. The extrapolation parameters of BMMe are up…
Deep Nonnegative Matrix Factorization with Beta Divergences
Valentin Leplat, Le Thi Khanh Hien, Akwum Onwunta +1
Deep Nonnegative Matrix Factorization (deep NMF) has recently emerged as a valuable technique for extracting multiple layers of features across different scales. However, all exist…