36 citations · 70 across the 11 of their papers we have counts for
11 papers
Minority Takeover in Majority Dynamics: Searching for Rare Initializations via the History Passing Algorithm
Marek Jankola, Freya Behrens, Cédric Koller +1
We investigate how much bias in the initial configuration is required to drive global agreement in synchronous, deterministic majority dynamics on large random -regular graphs.…
Dataset distillation for memorized data: Soft labels can leak held-out teacher knowledge
Freya Behrens, Lenka Zdeborová
Dataset distillation aims to compress training data into fewer examples via a teacher, from which a student can learn effectively. While its success is often attributed to structur…
Dynamical Cavity Method for Hypergraphs and its Application to Quenches in the k-XOR-SAT Problem
Aude Maier, Freya Behrens, Lenka Zdeborová
The dynamical cavity method and its backtracking version provide a powerful approach to studying the properties of dynamical processes on large random graphs. This paper extends th…
Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
Beatriz Borges, Negar Foroutan, Deniz Bayazit +87
AI assistants are being increasingly used by students enrolled in higher education institutions. While these tools provide opportunities for improved teaching and education, they a…
Counting in Small Transformers: The Delicate Interplay between Attention and Feed-Forward Layers
Freya Behrens, Luca Biggio, Lenka Zdeborová
Next to scaling considerations, architectural design choices profoundly shape the solution space of transformers. In this work, we analyze the solutions simple transformer blocks i…
Counting and Hardness-of-Finding Fixed Points in Cellular Automata on Random Graphs
Cédric Koller, Freya Behrens, Lenka Zdeborová
We study the fixed points of outer-totalistic cellular automata on sparse random regular graphs. These can be seen as constraint satisfaction problems, where each variable must adh…
Quenches in the Sherrington-Kirkpatrick model
Vittorio Erba, Freya Behrens, Florent Krzakala +1
The Sherrington-Kirkpatrick (SK) model is a prototype of a complex non-convex energy landscape. Dynamical processes evolving on such landscapes and locally aiming to reach minima a…
A phase transition between positional and semantic learning in a solvable model of dot-product attention
Hugo Cui, Freya Behrens, Florent Krzakala +1
Many empirical studies have provided evidence for the emergence of algorithmic mechanisms (abilities) in the learning of language models, that lead to qualitative improvements of t…
Dynamical Phase Transitions in Graph Cellular Automata
Freya Behrens, Barbora Hudcová, Lenka Zdeborová
Discrete dynamical systems can exhibit complex behaviour from the iterative application of straightforward local rules. A famous example are cellular automata whose global dynamics…
(Dis)assortative Partitions on Random Regular Graphs
Freya Behrens, Gabriel Arpino, Yaroslav Kivva +1
We study the problem of assortative and disassortative partitions on random -regular graphs. Nodes in the graph are partitioned into two non-empty groups. In the assortative par…
Neurally Augmented ALISTA
Freya Behrens, Jonathan Sauder, Peter Jung
It is well-established that many iterative sparse reconstruction algorithms can be unrolled to yield a learnable neural network for improved empirical performance. A prime example…