11 citations · 32 across the 41 of their papers we have counts for
14 papers · 1 filter
Topological recursion for spin enumeration of hypermaps
Alexander Alexandrov, Boris Bychkov, Petr Dunin-Barkowski +2
We establish topological recursion for the spin enumeration of hypermaps, or, in the dual language, spin dessins d'enfants. This problem has remarkable features distinguishing it f…
Decision Models vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places
Delip Rao, Chris Callison-Burch
LLM judges now score outputs against rubrics, in benchmarks and as training rewards. Decision models return probabilities over permitted answers with a confidence score but no free…
EviStreams: Human-in-the-Loop AI Data Extraction for Systematic Reviews in Medicine
Sai Karthik Kosuri, Ankita Shashikant Bhosale, Michael Glick +2
Systematic reviews underpin clinical guidelines, yet their data-extraction step is a major expert-labor bottleneck bound by a protocolized workflow: two reviewers extract each stud…
Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning
Ajay Patel, Kartik Hosanagar, Ramayya Krishnan +2
Large language models (LLMs) are improving rapidly as reflected in benchmark scores, yet these AI benchmarks largely test capabilities such as factual recall, narrow question answe…
Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why
Delip Rao, Chris Callison-Burch
Whether a rubric-based LLM judge can replace human annotation is decided by its measured agreement with human labels. Yet the same verdicts can support wildly varying agreement num…
PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Tasks
Bowen Jiang, Yuan Yuan, Zhuoqun Hao +12
Personal intelligence is becoming a central frontier for user-facing AI agents. To be helpful in everyday life, agents must understand users across the digital contexts where their…
When Verification Fails: How Compositionally Infeasible Claims Escape Rejection
Muxin Liu, Delip Rao, Grace Kim +1
Scientific claim verification, the task of determining whether claims are entailed by scientific evidence, is fundamental to establishing discoveries in evidence while preventing m…
Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents
Delip Rao, Eric Wong, Chris Callison-Burch
Large language models and deep research agents supply citation URLs to support their claims, yet the reliability of these citations has not been systematically measured. We address…
What Do Claim Verification Datasets Actually Test? A Reasoning Trace Analysis
Delip Rao, Chris Callison-Burch
Despite rapid progress in claim verification, we lack a systematic understanding of what reasoning these benchmarks actually exercise. We generate structured reasoning traces for 2…
ThinknCheck: Grounded Claim Verification with Compact, Reasoning-Driven, and Interpretable Models
Delip Rao, Feijiang Han, Chris Callison-Burch
We present ThinknCheck, a 1B-parameter verifier for grounded claim verification that first produces a short, structured rationale and then a binary verdict. We construct LLMAggreFa…
BibTeX Citation Errors in Scientific Publishing Agents: Evaluation and Mitigation
Delip Rao, Chris Callison-Burch
Large language models with web search are increasingly used in scientific publishing agents, yet they produce BibTeX entries with pervasive field-level errors stemming from omissio…
Autorubric: A Unifying Framework for Rubric-Based LLM Evaluation on Non-Verifiable Tasks
Delip Rao, Chris Callison-Burch
Rubric-based LLM judges have become indispensable for evaluating and optimizing systems on non-verifiable tasks, where success cannot be reduced to exact programmatic checks. Yet t…
Reliable and Responsible Foundation Models: A Comprehensive Survey
Xinyu Yang, Junlin Han, Rishi Bommasani +49
Foundation models, including Large Language Models (LLMs), Multimodal Large Language Models (MLLMs), Image Generative Models (i.e, Text-to-Image Models and Image-Editing Models), a…
FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
Ajay Patel, Colin Raffel, Chris Callison-Burch
Due to limited supervised training data, large language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstruct…