980 citations · 2k across the 73 of their papers we have counts for
42 papers · 1 filter
Information-theoretic Characterizations of Generalization Error for the Gibbs Algorithm
Gholamali Aminian, Yuheng Bu, Laura Toni +2
Various approaches have been developed to upper bound the generalization error of a supervised learning algorithm. However, existing bounds are often loose and even vacuous when ev…
Tighter Expected Generalization Error Bounds via Convexity of Information Measures
Gholamali Aminian, Yuheng Bu, Gregory Wornell +1
Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average jo…
Blind Modulo Analog-to-Digital Conversion of Vector Processes
Amir Weiss, Everest Huang, Or Ordentlich +1
In a growing number of applications, there is a need to digitize a (possibly high) number of correlated signals whose spectral characteristics are challenging for traditional analo…
Super-Nyquist Rateless Coding for Intersymbol Interference Channels
Uri Erez, Gregory W. Wornell
A rateless transmission architecture is developed for communication over Gaussian intersymbol interference channels, based on the concept of super-Nyquist (SNQ) signaling. In such…
An Information Theoretic Interpretation to Deep Neural Networks
Shao-Lun Huang, Xiangxiang Xu, Lizhong Zheng +1
It is commonly believed that the hidden layers of deep neural networks (DNNs) attempt to extract informative features for learning tasks. In this paper, we formalize this intuition…
Bregman Divergence Bounds and Universality Properties of the Logarithmic Loss
Amichai Painsky, Gregory W. Wornell
A loss function measures the discrepancy between the true values and their estimated fits, for a given instance of data. In classification problems, a loss function is said to be p…