78 citations · 130 across the 30 of their papers we have counts for
31 papers
Information-Theoretic Classifier-Free Guidance with Adaptive Schedule Optimization
Haobo Chen, Xiangxiang Xu, Yuheng Bu
Diffusion models have achieved strong performance in image, text-to-image, and video generation, where conditional generation is often controlled by classifier-free guidance (CFG).…
Information-theoretic Analysis of the Gibbs Algorithm: An Individual Sample Approach
Youheng Zhu, Yuheng Bu
Recent progress has shown that the generalization error of the Gibbs algorithm can be exactly characterized using the symmetrized KL information between the learned hypothesis and…
An Algorithm for Computing the Capacity of Symmetrized KL Information for Discrete Channels
Haobo Chen, Gholamali Aminian, Yuheng Bu
Symmetrized Kullback-Leibler (KL) information (\(I_{\mathrm{SKL}}\)), which symmetrizes the traditional mutual information by integrating Lautum information, has been shown as a cr…
Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage?
Maohao Shen, J. Jon Ryu, Soumya Ghosh +4
This paper questions the effectiveness of a modern predictive uncertainty quantification approach, called \emph{evidential deep learning} (EDL), in which a single neural network mo…
Operator SVD with Neural Networks via Nested Low-Rank Approximation
J. Jon Ryu, Xiangxiang Xu, H. S. Melihcan Erol +3
Computing eigenvalue decomposition (EVD) of a given linear operator, or finding its leading eigenvalues and eigenfunctions, is a fundamental task in many machine learning and scien…
Gibbs-Based Information Criteria and the Over-Parameterized Regime
Haobo Chen, Yuheng Bu, Gregory W. Wornell
Double-descent refers to the unexpected drop in test loss of a learning algorithm beyond an interpolating threshold with over-parameterization, which is not predicted by informatio…
Feature Learning in Image Hierarchies using Functional Maximal Correlation
Bo Hu, Yuheng Bu, José C. Príncipe
This paper proposes the Hierarchical Functional Maximal Correlation Algorithm (HFMCA), a hierarchical methodology that characterizes dependencies across two hierarchical levels in…
A Bilateral Bound on the Mean-Square Error for Estimation in Model Mismatch
Amir Weiss, Alejandro Lancho, Yuheng Bu +1
A bilateral (i.e., upper and lower) bound on the mean-square error under a general model mismatch is developed. The bound, which is derived from the variational representation of t…
Reliable Gradient-free and Likelihood-free Prompt Tuning
Maohao Shen, Soumya Ghosh, Prasanna Sattigeri +3
Due to privacy or commercial constraints, large pre-trained language models (PLMs) are often offered as black-box APIs. Fine-tuning such models to downstream tasks is challenging b…
On the Generalization Error of Meta Learning for the Gibbs Algorithm
Yuheng Bu, Harsha Vardhan Tetali, Gholamali Aminian +2
We analyze the generalization ability of joint-training meta learning algorithms via the Gibbs algorithm. Our exact characterization of the expected meta generalization error for t…
Group Fairness with Uncertainty in Sensitive Attributes
Abhin Shah, Maohao Shen, Jongha Jon Ryu +4
Learning a fair predictive model is crucial to mitigate biased decisions against minority groups in high-stakes applications. A common approach to learn such a model involves solvi…
Post-hoc Uncertainty Learning using a Dirichlet Meta-Model
Maohao Shen, Yuheng Bu, Prasanna Sattigeri +3
It is known that neural networks have the problem of being over-confident when directly using the output label distribution to generate uncertainty measures. Existing methods mainl…
Information-theoretic Characterizations of Generalization Error for the Gibbs Algorithm
Gholamali Aminian, Yuheng Bu, Laura Toni +2
Various approaches have been developed to upper bound the generalization error of a supervised learning algorithm. However, existing bounds are often loose and even vacuous when ev…
How Does Pseudo-Labeling Affect the Generalization Error of the Semi-Supervised Gibbs Algorithm?
Haiyun He, Gholamali Aminian, Yuheng Bu +2
We provide an exact characterization of the expected generalization error (gen-error) for semi-supervised learning (SSL) with pseudo-labeling via the Gibbs algorithm. The gen-error…
Data-Driven Blind Synchronization and Interference Rejection for Digital Communication Signals
Alejandro Lancho, Amir Weiss, Gary C. F. Lee +4
We study the potential of data-driven deep learning methods for separation of two communication signals from an observation of their mixture. In particular, we assume knowledge on…
Exploiting Temporal Structures of Cyclostationary Signals for Data-Driven Single-Channel Source Separation
Gary C. F. Lee, Amir Weiss, Alejandro Lancho +4
We study the problem of single-channel source separation (SCSS), and focus on cyclostationary signals, which are particularly suitable in a variety of application domains. Unlike c…
Tighter Expected Generalization Error Bounds via Convexity of Information Measures
Gholamali Aminian, Yuheng Bu, Gregory Wornell +1
Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average jo…
On Balancing Bias and Variance in Unsupervised Multi-Source-Free Domain Adaptation
Maohao Shen, Yuheng Bu, Gregory Wornell
Due to privacy, storage, and other constraints, there is a growing need for unsupervised domain adaptation techniques in machine learning that do not require access to the data use…
Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm
Yuheng Bu, Gholamali Aminian, Laura Toni +2
We provide an information-theoretic analysis of the generalization ability of Gibbs-based transfer learning algorithms by focusing on two popular transfer learning approaches, -…
Selective Regression Under Fairness Criteria
Abhin Shah, Yuheng Bu, Joshua Ka-Wing Lee +4
Selective regression allows abstention from prediction if the confidence to make an accurate prediction is not sufficient. In general, by allowing a reject option, one expects the…
Characterizing the Generalization Error of Gibbs Algorithm with Symmetrized KL information
Gholamali Aminian, Yuheng Bu, Laura Toni +2
Bounding the generalization error of a supervised learning algorithm is one of the most important problems in learning theory, and various approaches have been developed. However,…
A Maximal Correlation Approach to Imposing Fairness in Machine Learning
Joshua Lee, Yuheng Bu, Prasanna Sattigeri +4
As machine learning algorithms grow in popularity and diversify to many industries, ethical and legal concerns regarding their fairness have become increasingly relevant. We explor…
Adaptive Sequential Machine Learning
Craig Wilson, Yuheng Bu, Venugopal Veeravalli
A framework previously introduced in [3] for solving a sequence of stochastic optimization problems with bounded changes in the minimizers is extended and applied to machine learni…
Information-Theoretic Understanding of Population Risk Improvement with Model Compression
Yuheng Bu, Weihao Gao, Shaofeng Zou +1
We show that model compression can improve the population risk of a pre-trained model, by studying the tradeoff between the decrease in the generalization error and the increase in…