57 citations · 432 across the 81 of their papers we have counts for
7 papers · 2 filters
Towards Statistical and Computational Complexities of Polyak Step Size Gradient Descent
Tongzheng Ren, Fuheng Cui, Alexia Atsidakou +2
We study the statistical and computational complexities of the Polyak step size gradient descent algorithm under generalized smoothness and Lojasiewicz conditions of the population…
On Cross-Layer Alignment for Model Fusion of Heterogeneous Neural Networks
Dang Nguyen, Trang Nguyen, Khai Nguyen +3
Layer-wise model fusion via optimal transport, named OTFusion, applies soft neuron association for unifying different pre-trained networks to save computational resources. While en…
On Label Shift in Domain Adaptation via Wasserstein Distance
Trung Le, Dat Do, Tuan Nguyen +4
We study the label shift problem between the source and target domains in general domain adaptation (DA) settings. We consider transformations transporting the target to source dom…
Improving Transformers with Probabilistic Attention Keys
Tam Nguyen, Tan M. Nguyen, Dung D. Le +5
Multi-head attention is a driving force behind state-of-the-art transformers, which achieve remarkable performance across a variety of natural language processing (NLP) and compute…
Structured Dropout Variational Inference for Bayesian Neural Networks
Son Nguyen, Duong Nguyen, Khai Nguyen +3
Approximate inference in Bayesian deep networks exhibits a dilemma of how to yield high fidelity posterior approximations while maintaining computational efficiency and scalability…
On Robust Optimal Transport: Computational Complexity and Barycenter Computation
Khang Le, Huy Nguyen, Quang Nguyen +3
We consider robust variants of the standard optimal transport, named robust optimal transport, where marginal constraints are relaxed via Kullback-Leibler divergence. We show that…
On the computational and statistical complexity of over-parameterized matrix sensing
Jiacheng Zhuo, Jeongyeol Kwon, Nhat Ho +1
We consider solving the low rank matrix sensing problem with Factorized Gradient Descend (FGD) method when the true rank is unknown and over-specified, which we refer to as over-pa…