Next-Token Prediction Task Assumes Optimal Data Ordering for LLM Training in Proof Generation

Chenyang An, Shima Imani, Feng Yao +8

In the field of large language model (LLM)-based proof generation, despite extensive training on large datasets such as ArXiv, LLMs still exhibit only modest performance on proving…