57 citations · 432 across the 81 of their papers we have counts for
29 papers · 1 filter
Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
Huy Nguyen, Pedram Akbarian, Fanqi Yan +1
Top-K sparse softmax gating mixture of experts has been widely used for scaling up massive deep-learning architectures without increasing the computational cost. Despite its popula…
Quasi-Monte Carlo for 3D Sliced Wasserstein
Khai Nguyen, Nicola Bariletto, Nhat Ho
Monte Carlo (MC) integration has been employed as the standard approximation method for the Sliced Wasserstein (SW) distance, whose analytical expression involves an intractable ex…
Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoders
Hien Dang, Tho Tran, Tan Nguyen +1
The posterior collapse phenomenon in variational autoencoder (VAE), where the variational posterior distribution closely matches the prior distribution, can hinder the quality of t…
Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts
Huy Nguyen, TrungTin Nguyen, Khai Nguyen +1
Originally introduced as a neural network for ensemble learning, mixture of experts (MoE) has recently become a fundamental building block of highly successful modern deep neural n…
Demystifying Softmax Gating Function in Gaussian Mixture of Experts
Huy Nguyen, TrungTin Nguyen, Nhat Ho
Understanding the parameter estimation of softmax gating Gaussian mixture of experts has remained a long-standing open problem in the literature. It is mainly due to three fundamen…
Sliced Wasserstein Estimation with Control Variates
Khai Nguyen, Nhat Ho
The sliced Wasserstein (SW) distances between two probability measures are defined as the expectation of the Wasserstein distance between two one-dimensional projections of the two…
Energy-Based Sliced Wasserstein Distance
Khai Nguyen, Nhat Ho
The sliced Wasserstein (SW) distance has been widely recognized as a statistically effective and computationally efficient metric between two probability measures. A key component…
Markovian Sliced Wasserstein Distances: Beyond Independent Projections
Khai Nguyen, Tongzheng Ren, Nhat Ho
Sliced Wasserstein (SW) distance suffers from redundant projections due to independent uniform random projecting directions. To partially overcome the issue, max K sliced Wasserste…
Fast Approximation of the Generalized Sliced-Wasserstein Distance
Dung Le, Huy Nguyen, Khai Nguyen +2
Generalized sliced Wasserstein distance is a variant of sliced Wasserstein distance that exploits the power of non-linear projection through a given defining function to better cap…
Hierarchical Sliced Wasserstein Distance
Khai Nguyen, Tongzheng Ren, Huy Nguyen +3
Sliced Wasserstein (SW) distance has been widely used in different application scenarios since it can be scaled to a large number of supports without suffering from the curse of di…
Beyond EM Algorithm on Over-specified Two-Component Location-Scale Gaussian Mixtures
Tongzheng Ren, Fuheng Cui, Sujay Sanghavi +1
The Expectation-Maximization (EM) algorithm has been predominantly used to approximate the maximum likelihood estimation of the location-scale Gaussian mixtures. However, when the…
An Exponentially Increasing Step-size for Parameter Estimation in Statistical Models
Nhat Ho, Tongzheng Ren, Sujay Sanghavi +2
Using gradient descent (GD) with fixed or decaying step-size is a standard practice in unconstrained optimization problems. However, when the loss function is only locally convex,…
Amortized Projection Optimization for Sliced Wasserstein Generative Models
Khai Nguyen, Nhat Ho
Seeking informative projecting directions has been an important task in utilizing sliced Wasserstein distance in applications. However, finding these directions usually requires an…
Improving Computational Complexity in Statistical Models with Second-Order Information
Tongzheng Ren, Jiacheng Zhuo, Sujay Sanghavi +1
It is known that when the statistical models are singular, i.e., the Fisher information matrix at the true parameter is degenerate, the fixed step-size gradient descent algorithm t…
Beyond Black Box Densities: Parameter Learning for the Deviated Components
Dat Do, Nhat Ho, XuanLong Nguyen
As we collect additional samples from a data population for which a known density function estimate may have been previously obtained by a black box method, the increased complexit…
Improving Mini-batch Optimal Transport via Partial Transportation
Khai Nguyen, Dang Nguyen, The-Anh Vu-Le +2
Mini-batch optimal transport (m-OT) has been widely used recently to deal with the memory issue of OT in large-scale applications. Despite their practicality, m-OT suffers from mis…
On Multimarginal Partial Optimal Transport: Equivalent Forms and Computational Complexity
Khang Le, Huy Nguyen, Tung Pham +1
We study the multi-marginal partial optimal transport (POT) problem between discrete (unbalanced) measures with at most supports. We first prove that we can obtain two equi…
On Transportation of Mini-batches: A Hierarchical Approach
Khai Nguyen, Dang Nguyen, Quoc Nguyen +5
Mini-batch optimal transport (m-OT) has been successfully used in practical applications that involve probability measures with a very high number of supports. The m-OT solves seve…
Improving Relational Regularized Autoencoders with Spherical Sliced Fused Gromov Wasserstein
Khai Nguyen, Son Nguyen, Nhat Ho +2
Relational regularized autoencoder (RAE) is a framework to learn the distribution of data by minimizing a reconstruction loss together with a relational regularization on the laten…
On the Minimax Optimality of the EM Algorithm for Learning Two-Component Mixed Linear Regression
Jeongyeol Kwon, Nhat Ho, Constantine Caramanis
We study the convergence rates of the EM algorithm for learning two-component mixed linear regression under all regimes of signal-to-noise ratio (SNR). We resolve a long-standing q…
Distributional Sliced-Wasserstein and Applications to Generative Modeling
Khai Nguyen, Nhat Ho, Tung Pham +1
Sliced-Wasserstein distance (SW) and its variant, Max Sliced-Wasserstein distance (Max-SW), have been used widely in the recent years due to their fast computation and scalability…
Sampling for Bayesian Mixture Models: MCMC with Polynomial-Time Mixing
Wenlong Mou, Nhat Ho, Martin J. Wainwright +2
We study the problem of sampling from the power posterior distribution in Bayesian Gaussian mixture models, a robust version of the classical posterior. This power posterior is kno…
Tree-Wasserstein Barycenter for Large-Scale Multilevel Clustering and Scalable Bayes
Tam Le, Viet Huynh, Nhat Ho +2
We study in this paper a variant of Wasserstein barycenter problem, which we refer to as tree-Wasserstein barycenter, by leveraging a specific class of ground metrics, namely tree…
Flow-based Alignment Approaches for Probability Measures in Different Spaces
Tam Le, Nhat Ho, Makoto Yamada
Gromov-Wasserstein (GW) is a powerful tool to compare probability measures whose supports are in different metric spaces. GW suffers however from a computational drawback since it…