9 citations · 39 across the 37 of their papers we have counts for
7 papers · 1 filter
Streaming Sliced Optimal Transport
Khai Nguyen
Sliced optimal transport (SOT), or sliced Wasserstein (SW) distance, is widely recognized for its statistical and computational scalability. In this work, we further enhance comput…
Improving Routing in Sparse Mixture of Experts with Graph of Tokens
Tam Nguyen, Ngoc N. Tran, Khai Nguyen +1
Sparse Mixture of Experts (SMoE) has emerged as a key to achieving unprecedented scalability in deep learning. By activating only a small subset of parameters per sample, SMoE achi…
Lightspeed Geometric Dataset Distance via Sliced Optimal Transport
Khai Nguyen, Hai Nguyen, Tuan Pham +1
We introduce sliced optimal transport dataset distance (s-OTDD), a model-agnostic, embedding-agnostic approach for dataset comparison that requires no training, is robust to variat…
Designing Robust Transformers using Robust Kernel Density Estimation
Xing Han, Tongzheng Ren, Tan Minh Nguyen +3
Recent advances in Transformer architectures have empowered their empirical success in a variety of tasks across different domains. However, existing works mainly focus on predicti…
Transformer with Fourier Integral Attentions
Tan Nguyen, Minh Pham, Tam Nguyen +3
Multi-head attention empowers the recent success of transformers, the state-of-the-art models that have achieved remarkable success in sequence modeling and beyond. These attention…
On Cross-Layer Alignment for Model Fusion of Heterogeneous Neural Networks
Dang Nguyen, Trang Nguyen, Khai Nguyen +3
Layer-wise model fusion via optimal transport, named OTFusion, applies soft neuron association for unifying different pre-trained networks to save computational resources. While en…
Structured Dropout Variational Inference for Bayesian Neural Networks
Son Nguyen, Duong Nguyen, Khai Nguyen +3
Approximate inference in Bayesian deep networks exhibits a dilemma of how to yield high fidelity posterior approximations while maintaining computational efficiency and scalability…