36 citations · 70 across the 11 of their papers we have counts for
6 papers · 1 filter
Dynamical Cavity Method for Hypergraphs and its Application to Quenches in the k-XOR-SAT Problem
Aude Maier, Freya Behrens, Lenka Zdeborová
The dynamical cavity method and its backtracking version provide a powerful approach to studying the properties of dynamical processes on large random graphs. This paper extends th…
Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
Beatriz Borges, Negar Foroutan, Deniz Bayazit +87
AI assistants are being increasingly used by students enrolled in higher education institutions. While these tools provide opportunities for improved teaching and education, they a…
Counting in Small Transformers: The Delicate Interplay between Attention and Feed-Forward Layers
Freya Behrens, Luca Biggio, Lenka Zdeborová
Next to scaling considerations, architectural design choices profoundly shape the solution space of transformers. In this work, we analyze the solutions simple transformer blocks i…
Counting and Hardness-of-Finding Fixed Points in Cellular Automata on Random Graphs
Cédric Koller, Freya Behrens, Lenka Zdeborová
We study the fixed points of outer-totalistic cellular automata on sparse random regular graphs. These can be seen as constraint satisfaction problems, where each variable must adh…
Quenches in the Sherrington-Kirkpatrick model
Vittorio Erba, Freya Behrens, Florent Krzakala +1
The Sherrington-Kirkpatrick (SK) model is a prototype of a complex non-convex energy landscape. Dynamical processes evolving on such landscapes and locally aiming to reach minima a…
A phase transition between positional and semantic learning in a solvable model of dot-product attention
Hugo Cui, Freya Behrens, Florent Krzakala +1
Many empirical studies have provided evidence for the emergence of algorithmic mechanisms (abilities) in the learning of language models, that lead to qualitative improvements of t…