57 citations · 432 across the 81 of their papers we have counts for
23 papers · 1 filter
A Primal-Dual Framework for Transformers and Neural Networks
Tan M. Nguyen, Tam Nguyen, Nhat Ho +3
Self-attention is key to the remarkable success of transformers in sequence modeling tasks including many applications in natural language processing and computer vision. Like neur…
Privacy Preserving Bayesian Federated Learning in Heterogeneous Settings
Disha Makhija, Joydeep Ghosh, Nhat Ho
In several practical applications of federated learning (FL), the clients are highly heterogeneous in terms of both their data and compute resources, and therefore enforcing the sa…
Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Data
Hien Dang, Tho Tran, Stanley Osher +3
Modern deep neural networks have achieved impressive performance on tasks from image classification to natural language processing. Surprisingly, these complex systems with massive…
Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature
Khang Nguyen, Hieu Nong, Vinh Nguyen +3
Graph Neural Networks (GNNs) had been demonstrated to be inherently susceptible to the problems of over-smoothing and over-squashing. These issues prohibit the ability of GNNs to m…
Beyond Losses Reweighting: Empowering Multi-Task Learning via the Generalization Perspective
Hoang Phan, Lam Tran, Quyen Tran +6
Multi-task learning (MTL) trains deep neural networks to optimize several objectives simultaneously using a shared backbone, which leads to reduced computational costs, improved da…
Designing Robust Transformers using Robust Kernel Density Estimation
Xing Han, Tongzheng Ren, Tan Minh Nguyen +3
Recent advances in Transformer architectures have empowered their empirical success in a variety of tasks across different domains. However, existing works mainly focus on predicti…
Improving Generative Flow Networks with Path Regularization
Anh Do, Duy Dinh, Tan Nguyen +3
Generative Flow Networks (GFlowNets) are recently proposed models for learning stochastic policies that generate compositional objects by sequences of actions with the probability…
Stochastic Multiple Target Sampling Gradient Descent
Hoang Phan, Ngoc Tran, Trung Le +3
Sampling from an unnormalized target distribution is an essential problem with many applications in probabilistic inference. Stein Variational Gradient Descent (SVGD) has been show…
Transformer with Fourier Integral Attentions
Tan Nguyen, Minh Pham, Tam Nguyen +3
Multi-head attention empowers the recent success of transformers, the state-of-the-art models that have achieved remarkable success in sequence modeling and beyond. These attention…
Federated Self-supervised Learning for Heterogeneous Clients
Disha Makhija, Nhat Ho, Joydeep Ghosh
Federated Learning has become an important learning paradigm due to its privacy and computational benefits. As the field advances, two key challenges that still remain to be addres…
Efficient Forecasting of Large Scale Hierarchical Time Series via Multilevel Clustering
Xing Han, Tongzheng Ren, Jing Hu +2
We propose a novel approach to the problem of clustering hierarchically aggregated time-series data, which has remained an understudied problem though it has several commercial app…
Global-Local Regularization Via Distributional Robustness
Hoang Phan, Trung Le, Trung Phung +3
Despite superior performance in many situations, deep neural networks are often vulnerable to adversarial examples and distribution shifts, limiting model generalization ability in…
Architecture Agnostic Federated Learning for Neural Networks
Disha Makhija, Xing Han, Nhat Ho +1
With growing concerns regarding data privacy and rapid increase in data volume, Federated Learning(FL) has become an important learning paradigm. However, jointly learning a deep n…
Towards Statistical and Computational Complexities of Polyak Step Size Gradient Descent
Tongzheng Ren, Fuheng Cui, Alexia Atsidakou +2
We study the statistical and computational complexities of the Polyak step size gradient descent algorithm under generalized smoothness and Lojasiewicz conditions of the population…
On Cross-Layer Alignment for Model Fusion of Heterogeneous Neural Networks
Dang Nguyen, Trang Nguyen, Khai Nguyen +3
Layer-wise model fusion via optimal transport, named OTFusion, applies soft neuron association for unifying different pre-trained networks to save computational resources. While en…
On Label Shift in Domain Adaptation via Wasserstein Distance
Trung Le, Dat Do, Tuan Nguyen +4
We study the label shift problem between the source and target domains in general domain adaptation (DA) settings. We consider transformations transporting the target to source dom…
Improving Transformers with Probabilistic Attention Keys
Tam Nguyen, Tan M. Nguyen, Dung D. Le +5
Multi-head attention is a driving force behind state-of-the-art transformers, which achieve remarkable performance across a variety of natural language processing (NLP) and compute…
Structured Dropout Variational Inference for Bayesian Neural Networks
Son Nguyen, Duong Nguyen, Khai Nguyen +3
Approximate inference in Bayesian deep networks exhibits a dilemma of how to yield high fidelity posterior approximations while maintaining computational efficiency and scalability…
On Robust Optimal Transport: Computational Complexity and Barycenter Computation
Khang Le, Huy Nguyen, Quang Nguyen +3
We consider robust variants of the standard optimal transport, named robust optimal transport, where marginal constraints are relaxed via Kullback-Leibler divergence. We show that…
On the computational and statistical complexity of over-parameterized matrix sensing
Jiacheng Zhuo, Jeongyeol Kwon, Nhat Ho +1
We consider solving the low rank matrix sensing problem with Factorized Gradient Descend (FGD) method when the true rank is unknown and over-specified, which we refer to as over-pa…
Projection Robust Wasserstein Distance and Riemannian Optimization
Tianyi Lin, Chenyou Fan, Nhat Ho +2
Projection robust Wasserstein (PRW) distance, or Wasserstein projection pursuit (WPP), is a robust variant of the Wasserstein distance. Recent work suggests that this quantity is m…
Instability, Computational Efficiency and Statistical Accuracy
Nhat Ho, Koulik Khamaru, Raaz Dwivedi +3
Many statistical estimators are defined as the fixed point of a data-dependent operator, with estimators based on minimizing a cost function being an important special case. The li…
Probabilistic Multilevel Clustering via Composite Transportation Distance
Nhat Ho, Viet Huynh, Dinh Phung +1
We propose a novel probabilistic approach to multilevel clustering problems based on composite transportation distance, which is a variant of transportation distance where the unde…