1 citations · 2 across the 9 of their papers we have counts for
5 papers · 2 filters
Approximate Solomonoff Induction: On the Robustness of Learned Universal Predictors
Qian Li, Shang-Hua Teng
Why large language models trained via log-loss minimization over massive and diverse datasets exhibit broad capabilities and generalizable skills is not yet fully understood. Grau-…
Algorithmic Principles For Multiclass Learning Are Hard To Come By: Limits of Regularization and Proper Learning
Julian Asilis, Shaddin Dughmi, Vatsal Sharan +3
Two of the most fundamental questions in statistical learning theory are the following: which prediction problems are learnable, and how should they be learned? For the former, ele…
Prediction Under Imperfect Compression: A Theory of Approximate MDL
Qian Li, Xinyu Mao, Shang-Hua Teng +1
Minimum Description Length (MDL) formalizes the principle of Occam's razor by optimizing the total description length: . Fo…
Rethinking the Role of Positional Encoding: Sliding-Window Transformers without PE Remain Turing Complete
Qian Li, Xinyu Mao, Shang-Hua Teng
Positional encoding (PE) is widely viewed as necessary for transformers to process ordered sequences: without them, the next-token map appears permutation-invariant in its context…
A Theory of Time-Sensitive Language Generation: Sparse Hallucination Beats Mode Collapse
Atul Ganju, Travis McVoy, Shaddin Dughmi +1
We study language generation in the limit under a global preference ordering on strings, as introduced by Kleinberg and Wei. As is done in previous work, we aim for breadth, but im…