45 citations · 302 across the 119 of their papers we have counts for
28 papers · 1 filter
Verifier-Guided Self-Consistency: Test-Time Reasoning with Imperfect Generators and Verifiers
Ekaterina Fadeeva, Xueyan Li, Mrinmaya Sachan +1
Large language models can solve many multi-step reasoning problems, but their reasoning remains brittle: plausible intermediate steps often lead to incorrect answers. Improving rea…
DAPS: Dependency-Aware Premise Selection for LLM Theorem Proving
Zixuan Chen, Wenyuan Jiang, Junling Wang +2
LLM-based theorem proving in Lean 4 has advanced rapidly, but provers are difficult to leverage newly contributed lemmas, because retrieval skews toward popular foundational lemmas…
Same Beliefs, Different Policies: Learning Reusable Representations for Early Stopping in LLM Reasoning
Jingwei Ni, Dmitry Abulkhanov, Vladislav Smirnov +5
Reasoning LLMs often continue reasoning after reaching a correct answer, wasting computation and sometimes replacing it with an incorrect one. Deciding when to stop is a metareason…
SkillEval: Learning Interpretable Ability Profiles of LLMs via Cognitive Diagnosis Models
Berke Arda, Peng Cui, Qiaoyuan Zheng +3
Current evaluation of LLMs on benchmarks typically aggregates performance over individual examples into a single scalar score. This approach is overly coarse and ignores the divers…
CroissantMiner: Automated Extraction and Validation of Croissant Metadata for ML Datasets
Berke Arda, Ahmetcan Yavuz, Paul Gerry +7
Croissant has emerged as a standard for machine-readable dataset metadata, yet populating its fields remains labor-intensive and requires careful reading of accompanying dataset do…
The Assistance Dilemma: Learning to Teach via Multi-Turn Reinforcement Learning
Jakub Macina, Manu Kapur, Mrinmaya Sachan
Large language models (LLMs) trained to answer questions are natively poor at teaching. Reinforcement Learning (RL) against a simulated student is a promising approach to improve t…
What Makes Something Hard(er)? Explaining Question Difficulty in Natural Language
Peng Cui, Qiaoyuan Zheng, Rudolf Debelak +1
Difficulty is one of the most fundamental properties of a question: it determines whether the question can meaningfully discriminate between models of differing ability. Although a…
Do LLMs Exhibit Coherent Knowledge Structures in Mathematical Reasoning? A Perspective from Knowledge Space Theory
Peng Cui, Heejin Do, Mrinmaya Sachan
Human knowledge is inherently structured and interdependent: mastery of a concept requires prior mastery of its prerequisites, a principle formalized by Knowledge Space Theory (KST…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
SWIM: Student Writing Simulation via Proficiency-Conditioned Generation
Heejin Do, Jakub Kontak, Mrinmaya Sachan
Writing proficiency manifests in how students develop content, organize ideas, choose words, and use language. Despite growing interest in LLM-based student simulation, whether LLM…
Dynamically Allocating Evaluation Effort for Model Ranking
Vilém Zouhar, Julia Kreutzer, Alon Lavie +4
While human evaluation is the gold standard in many NLP tasks, it suffers from prohibitive costs and poor scalability. When identifying top-performing models, typical evaluation pr…
ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning
Vladislav Smirnov, Chieu Nguyen, Sergey Senichev +14
Test-time compute (TTC) scaling has emerged as a powerful paradigm for improving large language model (LLM) reasoning by allocating additional compute during inference, e.g., via m…
Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentation
Minjing Shi, Junling Wang, Jingwei Ni +2
Identifying logical fallacies in everyday discourse is challenging for many people. This challenge is amplified in the era of Large Language Models (LLMs), where malicious agents c…
Benchmarking and Enhancing Text-to-Image Models for Generating Visual Representations in Early Arithmetic Education
Junling Wang, Boqi Chen, Heejin Do +3
AI systems are increasingly used to support educational content creation, yet it remains unclear whether they can generate outputs that faithfully represent the pedagogical concept…
Diversity Matters: Revisiting Test-Time Compute in Vision-Language Models
Yijie Tong, Yifan Hou, Shaobo Cui +2
Test-time compute (TTC) strategies have emerged as a lightweight approach to boost reasoning in large language models (LLMs). However, their application and benefits for vision-lan…
Learning to Reason Efficiently with A* Post-Training
Andreas Opedal, Francesco Ignazio Re, Abulhair Saparov +3
Many applications of large language models (LLMs) require deductive reasoning, yet models frequently produce incorrect or redundant inference steps. We frame natural language infer…
Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators
Heejin Do, Shashank Sonkar, Mrinmaya Sachan
Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators. Y…
GeoDial: A Multimodal Conversational Tutoring Dataset for Geometry Problem-Solving with Visual Tutor Turns
Sankalan Pal Chowdhury, Junling Wang, Donya Rooein +2
Several educational domains rely heavily on diagrams and visual cues, yet most existing tutoring datasets are limited to text-only interactions. This limits the development of AI t…
Unveiling the Visual Counting Bottleneck in Vision-Language Models
Xingzhou Pang, Yifan Hou, Junling Wang +1
While Large Vision-Language Models (VLMs) excel at interpolation, they suffer catastrophic failures in systematic generalization, most notably in visual counting. In this work, we…
Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees
Xueyan Li, Johannes Zenn, Ekaterina Fadeeva +3
Self-consistency boosts inference-time performance by sampling multiple reasoning traces in parallel and voting. However, in constrained domains like math and code, this strategy i…
Misconception Acquisition Dynamics in Large Language Models
Naiming Liu, Xinghe Chen, Richard Baraniuk +2
Effective educational AI depends on modeling student misconceptions. Such models enable realistic learner simulation and diagnostic, adaptive tutoring. However, instruction-tuning…
Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation
Yanick Zengaffinen, Andreas Opedal, Donya Rooein +3
Modeling student misconceptions in a realistic manner is critical for AI in education. In this work, we examine how large language models (LLMs) reason about misconceptions when ge…
Post-Training Language Models for Crosslingual Consistency
Tianyu Liu, Jirui Qi, Mrinmaya Sachan +3
Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an i…
When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation
Mubashara Akhtar, Anka Reuel, Prajna Soni +34
Artificial intelligence benchmarks are an important mechanism to measure model progress and guide deployment decisions. However, benchmarks quickly "saturate", making it difficult…