Training Large Language Models To Reason In Parallel With Global Forking Tokens

Sheng Jia, Xiao Wang, Shiva Prasad Kasiviswanathan

Although LLMs have demonstrated improved performance by scaling parallel test-time compute, doing so relies on generating reasoning paths that are both diverse and accurate. For ch…