From the 1 of 3 linked papers with an AI index.
1 paper · 1 filter
Jacob Dunefsky, Wes Gurnee, Emmanuel Ameisen
The paper identifies a "countdown subcircuit" in Llama-3.1-70B-Instruct that compares the current token position to a target length, enabling the model to track remaining tokens fo…