5 papers
Minority Takeover in Majority Dynamics: Searching for Rare Initializations via the History Passing Algorithm
Marek Jankola, Freya Behrens, Cédric Koller +3
We investigate how much bias in the initial configuration is required to drive global agreement in synchronous, deterministic majority dynamics on large random -regular graphs.…
Dataset distillation for memorized data: Soft labels can leak held-out teacher knowledge
Freya Behrens, Lenka Zdeborová
Dataset distillation aims to compress training data into fewer examples via a teacher, from which a student can learn effectively. While its success is often attributed to structur…
Counting in Small Transformers: The Delicate Interplay between Attention and Feed-Forward Layers
Freya Behrens, Luca Biggio, Lenka Zdeborová
Next to scaling considerations, architectural design choices profoundly shape the solution space of transformers. In this work, we analyze the solutions simple transformer blocks i…
Dynamical Cavity Method for Hypergraphs and its Application to Quenches in the k-XOR-SAT Problem
Aude Maier, Freya Behrens, Lenka Zdeborová
The dynamical cavity method and its backtracking version provide a powerful approach to studying the properties of dynamical processes on large random graphs. This paper extends th…
Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
Beatriz Borges, Negar Foroutan, Deniz Bayazit +87
AI assistants are being increasingly used by students enrolled in higher education institutions. While these tools provide opportunities for improved teaching and education, they a…