6 papers
Frontier Coding Agents Use Metaprogramming to Adapt to Unfamiliar Programming Languages
Aman Sharma, Sushrut Thorat, Paras Chopra
LLM-based coding agents are usually evaluated in familiar software settings: mainstream languages, common libraries, and public repositories. These benchmarks remain important, but…
EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation
Aristotelis Lazaridis, Dylan Bates, Aman Sharma +3
On-Policy Distillation (OPD) has gained wide attraction as an LLM post-training paradigm due to its effectiveness in improving capabilities without introducing model distribution d…
EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages
Aman Sharma, Paras Chopra
Large language models achieve near-ceiling performance on code generation benchmarks, yet most of the programming languages used by popular benchmarks such as SWE-bench and HumanEv…
Measuring and Eliminating Refusals in Military Large Language Models
Jack FitzGerald, Dylan Bates, Aristotelis Lazaridis +17
Military Large Language Models (LLMs) must provide accurate information to the warfighter in time-critical and dangerous situations. However, today's LLMs are imbued with safety be…
The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Compute
Aman Sharma, Paras Chopra
We revisit test-time scaling for language model reasoning and ask a fundamental question: at equal token budget and compute, is it better to run multiple independent chains in para…
Think Just Enough: Sequence-Level Entropy as a Confidence Signal for LLM Reasoning
Aman Sharma, Paras Chopra
We introduce a simple, yet novel entropy-based framework to drive token efficiency in large language models during reasoning tasks. Our approach uses Shannon entropy from token-lev…