6 papers
MEMENTO: Teaching LLMs to Manage Their Own Context
Vasilis Kontonis, Yuchen Zeng, Shivam Garg +7
Reasoning models think in long, unstructured streams with no mechanism for compressing or organizing their own intermediate state. We introduce MEMENTO: a method that teaches model…
Wait, Wait, Wait... Why Do Reasoning Models Loop?
Charilaos Pipis, Shivam Garg, Vasilis Kontonis +3
Reasoning models (e.g., DeepSeek-R1) generate long chains of thought to solve harder problems, but they often loop, repeating the same text at low temperatures or with greedy decod…
Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions
Jaeyeon Kim, Kulin Shah, Vasilis Kontonis +2
In recent years, masked diffusion models (MDMs) have emerged as a promising alternative approach for generative modeling over discrete domains. Compared to autoregressive models (A…
Smoothed Analysis for Learning Concepts with Low Intrinsic Dimension
Gautam Chandrasekaran, Adam Klivans, Vasilis Kontonis +2
In traditional models of supervised learning, the goal of a learner -- given examples from an arbitrary joint distribution on -- is to output a hypo…
Learning Noisy Halfspaces with a Margin: Massart is No Harder than Random
Gautam Chandrasekaran, Vasilis Kontonis, Konstantinos Stavropoulos +1
We study the problem of PAC learning -margin halfspaces with Massart noise. We propose a simple proper learning algorithm, the Perspectron, that has sample complexity $\widetil…
Learning general Gaussian mixtures with efficient score matching
Sitan Chen, Vasilis Kontonis, Kulin Shah
We study the problem of learning mixtures of Gaussians in dimensions. We make no separation assumptions on the underlying mixture components: we only require that the covar…