7 papers
The Multilingual Artificial Hivemind: Do All LLMs Think Alike in Every Language?
Xaver Raphael Davey, Shubhra Mishra, Anton Adelöw +1
Text generated by Large Language Models (LLMs) often remains limited in diversity compared to that produced by humans. Recent advances demonstrate the presence of inter- and intra-…
DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluation
Vishal Kumar, Shubhra Mishra, Rebecca Hao +3
Large Language Models (LLMs) are increasingly being adopted as tools for learning; however, most tools remain text-only, limiting their usefulness for domains where visualizations…
A Matter of Interest: Understanding Interestingness of Math Problems in Humans and Language Models
Shubhra Mishra, Yuka Machino, Gabriel Poesia +9
The evolution of mathematics is shaped importantly by interestingness: researchers choose which problems to pursue, and students choose which problems to engage with, based on expe…
CurLL: A Developmental Framework to Evaluate Continual Learning in Language Models
Pavan Kalyan, Shubhra Mishra, Satya Lokam +1
We introduce a comprehensive continual learning dataset and benchmark (CurlL) grounded in human developmental trajectories from ages 5-10, enabling systematic and fine-grained asse…
From Next-Token to Mathematics: The Learning Dynamics of Mathematical Reasoning in Language Models
Shubhra Mishra, Gabriel Poesia, Noah D. Goodman
Large Language Models (LLMs) solely trained on next-token prediction learn to solve a wide range of problems involving mathematical reasoning. But how does this ability evolve duri…
An Evaluation Benchmark for Autoformalization in Lean4
Aryan Gulati, Devanshu Ladsaria, Shubhra Mishra +2
Large Language Models (LLMs) hold the potential to revolutionize autoformalization. The introduction of Lean4, a mathematical programming language, presents an unprecedented opport…
Families of Harris Graphs
Francesca Gandini, Shubhra Mishra, Douglas Shaw
A Harris Graph is a tough, Eulerian, non-Hamiltonian graph. Several approaches to creating new Harris graphs from existing ones are explored, including creating families of Harris…