5 papers
Learning Source Structure: Bayesian Prediction with Transformers
Meir Feder, Ruediger Urbanke
A transformer trained on sequences whose source parameters are drawn afresh each time can learn only what the sequences share: the source family and its prior. The Bayesian predict…
A Layered Simplex Architecture for Large Alphabets
Meir Feder, Yaniv Fogel, Ruediger Urbanke
A learning architecture induces a prior over its expressible models. Recent work links architectural success to a broad complexity spectrum: model probabilities spanning many order…
Reed-Muller Codes for Quantum Pauli and Multiple Access Channels
Dina Abdelhadi, Colin Sandon, Emmanuel Abbe +1
Reed-Muller (RM) codes have undergone significant analytical advancements over the past decade, particularly for binary memoryless symmetric (BMS) channels. We extend the scope of…
Interpolation of Quantum Polar Codes and Quantum Reed-Muller Codes
Keita Hidaka, Dina Abdelhadi, Ruediger Urbanke
Good quantum error-correcting codes that fulfill practical considerations, such as simple encoding circuits and efficient decoders, are essential for functional quantum information…
Information-Theoretic Framework for Understanding Modern Machine-Learning
Meir Feder, Ruediger Urbanke, Yaniv Fogel
We introduce an information-theoretic framework that views learning as universal prediction under log loss, characterized through regret bounds. Central to the framework is an effe…