4 papers
Re-executing Coding-Agent Evaluations: Unchanged Decisions and Unresolved Policy Risk
Tianyu Ding, Yaocheng Tong, Muzhi Yang +3
Executable tests grade large language model (LLM) coding-agent patches for leaderboards, rewards and self-improvement. Two preregistered re-execution studies returned negative resu…
Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
Tianyu Ding, Aditya Nannapaneni, Bingfan Liu +1
Large language model (LLM) agents are increasingly used across the scientific research lifecycle: ideation, literature search, experiment design and execution, analysis, manuscript…
Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgents
Tianyu Ding, Aditya Nannapaneni, Bingfan Liu +1
Always-on agents are systems whose future behavior depends on durable state accumulated across earlier interactions. We treat them as persistent-state systems: the operative system…
Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs
Tianyu Ding, Aditya Nannapaneni, Juan Pablo De la Cruz Weinstein
Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable. We audit eight automatic scorers -- lexical, embedding, and BERTSc…