65 citations · 281 across the 14 of their papers we have counts for
5 papers · 1 filter
Estimating Total Correlation with Mutual Information Estimators
Ke Bai, Pengyu Cheng, Weituo Hao +2
Total correlation (TC) is a fundamental concept in information theory that measures statistical dependency among multiple random variables. Recently, TC has shown noticeable effect…
MixKD: Towards Efficient Distillation of Large-scale Language Models
Kevin J Liang, Weituo Hao, Dinghan Shen +4
Large-scale language models have recently demonstrated impressive empirical performance. Nevertheless, the improved results are attained at the price of bigger models, more power c…
WAFFLe: Weight Anonymized Factorization for Federated Learning
Weituo Hao, Nikhil Mehta, Kevin J Liang +3
In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. In light of…
CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information
Pengyu Cheng, Weituo Hao, Shuyang Dai +3
Mutual information (MI) minimization has gained considerable interests in various machine learning tasks. However, estimating and minimizing MI in high-dimensional spaces remains a…
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training
Weituo Hao, Chunyuan Li, Xiujun Li +2
Learning to navigate in a visual environment following natural-language instructions is a challenging task, because the multimodal inputs to the agent are highly variable, and the…