4 papers
A Survey of Learn-to-Compute Paradigms for Rate-Distortion-Type Problems
Shitong Wu, Sicheng Xu, Lingyi Chen +4
Rate-distortion (RD) theory and its related formulations play a central role in understanding efficient information representation, but computing these quantities remains challengi…
A Gaussian Perspective for Distributional Discrepancy in Generative Diffusion Models
Qiang Sun, H. Vincent Poor, Wenyi Zhang
This paper introduces an analytical approach to quantifying and optimizing the distributional discrepancy in generative diffusion models. For a multivariate Gaussian source, we exp…
Tail-Aware Information-Theoretic Bounds for LLM Alignment under Heavy-Tailed Rewards
Huiming Zhang, Binghan Li, Wan Tian +1
Classical information-theoretic learning bounds typically rely on KL mutual information and moment-generating-function (MGF) arguments, which are well matched to bounded or sub-Gau…
Data-Driven Neural Estimation of Indirect Rate-Distortion Function
Zichao Yu, Qiang Sun, Wenyi Zhang
The rate-distortion function (RDF) has long been an information-theoretic benchmark for data compression. As its natural extension, the indirect rate-distortion function (iRDF) cor…