658 citations · 2.2k across the 187 of their papers we have counts for
18 papers · 1 filter
More of the same? Are scientific papers losing originality?
Ilan Doron-Arad, Elchanan Mossel
Scientific output is growing rapidly, but it is unclear whether the expanding literature remains original or is increasingly repeating itself. Originality has many dimensions. One…
Extremal subtrees of critical beta-splitting trees
Anna Brandenberger, Byron Chin, Elchanan Mossel
We determine the most and least likely shapes for an instance of the critical beta-splitting tree via a connection to data compression and Huffman's minimum redundancy codes. This…
Consensus times for monotone aggregation dynamics
Elchanan Mossel
We study an asynchronous consensus dynamics on agents: at each step a uniformly chosen agent replaces its state by , where is a fixed monotone aggregation…
Stochastic Autoregressive Learning
Ilan Doron-Arad, Idan Mehalel, Elchanan Mossel
Motivated by LLMs, which generate outputs by iteratively sampling from next-token distributions, we introduce a PAC-learning model for binary stochastic autoregressive learning. Th…
Recovering Assignments with One-Sided Noise
Cassandra Marcussen, Elchanan Mossel, Colin Sandon
We study the query complexity of recovering a planted assignment from a random constraint-satisfaction instance with one-sided noise. We consider the following 1-CNF recovery probl…
Denoising Distances in Metric Measure Spaces
Han Huang, Pakawut Jiradilok, Elchanan Mossel
Recent work studied the problem of finding clusters and denoising pairwise distances from noisy distances of points sampled on a manifold. We study the same problems in more genera…
Depth Lower Bounds for ReLU Networks with Binary Inputs
Neil Krishnan, Elchanan Mossel
We study the role of depth in ReLU networks with discrete (Boolean) inputs and real-valued outputs, complementing two established lines of work. For Boolean inputs, striking depth…
Genetic Algorithms with Optimization Guided Operators
Anna Brandenberger, Ilan Doron-Arad, Elchanan Mossel
Recent work in ML applies genetic algorithms at inference time to iteratively improve solutions to optimization problems. The basic mutation and recombination operators involved ar…
A Hierarchical Language Model with Predictable Scaling Laws and Provable Benefits of Reasoning
Jason Gaitonde, Frederic Koehler, Elchanan Mossel +2
We introduce a family of synthetic languages with hierarchical structure -- generated by a broadcast process on trees -- for which the role of context length and reasoning in autor…
The Benefits of Temporal Correlations: SGD Learns k-Juntas from Random Walks Efficiently
Elisabetta Cornacchia, Dan Mikulincer, Elchanan Mossel
We study how temporal correlations in the data can make certain sparse learning problems efficiently learnable by gradient-based methods. Our focus is on Boolean k-juntas, a canoni…
A Theory of Online Learning with Autoregressive Chain-of-Thought Reasoning
Ilan Doron-Arad, Idan Mehalel, Elchanan Mossel
Autoregressive generation lies at the heart of the mechanism of large language models. It can be viewed as the repeated application of a next-token generator: starting from an inpu…
Some Theoretical Limitations of t-SNE
Rupert Li, Elchanan Mossel
t-SNE has gained popularity as a dimension reduction technique, especially for visualizing data. It is well-known that all dimension reduction techniques may lose important feature…
Denoising distances beyond the volumetric barrier
Han Huang, Pakawut Jiradilok, Elchanan Mossel
We study the problem of reconstructing the latent geometry of a -dimensional Riemannian manifold from a random geometric graph. While recent works have made significant progress…
Sharp Threshold for the Convergence of Nonstationary Averaging
Saba Lepsveridze, Elchanan Mossel
We study non-stationary averaging processes, where each term of a sequence is a weighted average of previous terms, namely . Our results extend c…
Why ReLU? A Bit-Model Dichotomy for Deep Network Training
Ilan Doron-Arad, Elchanan Mossel
Theoretical analyses of Empirical Risk Minimization (ERM) are standardly framed within the Real-RAM model of computation. In this setting, training even simple neural networks is k…
Online Realizable Regression and Applications for ReLU Networks
Ilan Doron-Arad, Idan Mehalel, Elchanan Mossel
Realizable online regression can behave very differently from online classification. Even without any margin or stochastic assumptions, realizability may enforce horizon-free (fini…
Detecting Mutual Excitations in Non-Stationary Hawkes Processes
Elchanan Mossel, Anirudh Sridhar
We consider the problem of learning the network of mutual excitations (i.e., the dependency graph) in a non-stationary, multivariate Hawkes process. We consider a general setting w…
LLMs, Reasoning and Plagiarism
Elchanan Mossel
Recent reports claim that Large Language Models (LLMs) derive new science and exhibit human-level general intelligence. Such claims are entangled with two different narratives abou…