1 citations · 1 across the 20 of their papers we have counts for
12 papers · 1 filter
MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects
Jason Luo, Saibilila Abudukelimu, Judy Song +4
Document question-answering systems increasingly answer questions over collections of retrieved documents rather than one clean source, so robustness to distracting context matters…
Solving Is Not Drawing: A Benchmark for Diagrammatic Reasoning in Olympiad Geometry
Hsien Xin Peng, Anthony Kim, Alvin Li +3
Foundation models such as GPT and Claude now solve olympiad-level mathematics with remarkable proficiency, so much so that geometry problem solving has become a standard proxy for…
Diagnostic Foundation for Evaluating LLMs' Research Integrity as Co-Scientists
Yash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi +2
Language models are increasingly deployed as co-scientists, yet their ability to uphold research integrity under institutional pressure remains unmeasured. We introduce IntegrityBe…
Talked Out of the Truth: Sycophancy in the Reasoning Chains of Multimodal Models
Mahir Numayeer Islam, Gakuto Okuyama, Nikolaus Siauw +3
Large multimodal reasoning models (LMRMs) are increasingly capable, largely through generating explicit chain-of-thought reasoning before answering, but in language models this oft…
Poker Arena: Multi-Axis Profiling of Strategic Reasoning and Memory in LLMs
Pratham Singla, Shivank Garg, Vihan Singh
Strategic reasoning under uncertainty underpins consequential decisions in negotiation, finance, and policy, but prevailing game-play benchmarks collapse heterogeneous reasoning di…
Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark
Pratham Singla, Shivank Garg, Vihan Singh +1
Vision-language models (VLMs) are increasingly deployed where answers must follow from what is in the image, yet they often answer from textual priors, the question's phrasing toge…
Recurrent Reasoning on Symbolic Puzzles with Sequence Models
Gowrav Mannem, Chowdhury Marzia Mahjabin, Jason Chen +2
Large language models often appear strong on symbolic and algorithmic tasks, yet this apparent strength can hide brittle behaviour when problems become longer, harder, or slightly…
Text2Arch: A Dataset for Generating Scientific Architecture Diagrams from Natural Language Descriptions
Shivank Garg, Sankalp Mittal, Manish Gupta
Communicating complex system designs or scientific processes through text alone is inefficient and prone to ambiguity. A system that automatically generates scientific architecture…
When Prompt Optimization Becomes Jailbreaking: Adaptive Red-Teaming of Large Language Models
Zafir Shamsi, Nikhil Chekuru, Zachary Guzman +1
Large Language Models (LLMs) are increasingly integrated into high-stakes applications, making robust safety guarantees a central practical and commercial concern. Existing safety…
MiSCHiEF: A Benchmark in Minimal-Pairs of Safety and Culture for Holistic Evaluation of Fine-Grained Image-Caption Alignment
Sagarika Banerjee, Tangatar Madi, Advait Swaminathan +4
Fine-grained image-caption alignment is crucial for vision-language models (VLMs), especially in socially critical contexts such as identifying real-world risk scenarios or disting…
SIDiffAgent: Self-Improving Diffusion Agent
Shivank Garg, Ayush Singh, Gaurav Kumar Nayak
Text-to-image diffusion models have revolutionized generative AI, enabling high-quality and photorealistic image synthesis. However, their practical deployment remains hindered by…
ViT Registers and Fractal ViT
Jason Chuan-Chih Chou, Abhinav Kumar, Shivank Garg
Drawing inspiration from recent findings including surprisingly decent performance of transformers without positional encoding (NoPE) in the domain of language models and how regis…