5 papers
Generating Special Triangulations with Transformers
Charles Arnal, Jacky H. T. Yip, François Charton +1
Triangulations, i.e., well-structured decompositions of geometric objects into triangle-like pieces, are central objects in many domains of mathematics and physics. In particular,…
Transforming Calabi-Yau Constructions: Generating New Calabi-Yau Manifolds with Transformers
Jacky H. T. Yip, Charles Arnal, François Charton +1
Fine, regular, and star triangulations (FRSTs) of four-dimensional reflexive polytopes give rise to toric varieties, within which generic anticanonical hypersurfaces yield smooth C…
Iteration Head: A Mechanistic Study of Chain-of-Thought
Vivien Cabannes, Charles Arnal, Wassim Bouaziz +3
Chain-of-Thought (CoT) reasoning is known to improve Large Language Models both empirically and in terms of theoretical approximation power. However, our understanding of the inner…
Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification
Yunzhen Feng, Elvis Dohmatob, Pu Yang +2
Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because syn…
:Revealing the Decisive Effect of Instruction Diversity on Generalization
Dylan Zhang, Justin Wang, Francois Charton
Understanding and accurately following instructions is critical for large language models (LLMs) to be effective across diverse tasks. In this work, we rigorously examine the key f…