6 papers
The Last Human-Written Paper: Agent-Native Research Artifacts
Jiachen Liu, Jiaxin Pei, Jintao Huang +34
Scientific publication compresses a branching, iterative research process into a linear narrative, discarding the majority of what was discovered along the way. This compilation im…
The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Research
Xiaoyan Bai, Alexander Baumgartner, Haojia Sun +2
Reproducibility crises across sciences highlight the limitations of the paper-centric review system in assessing the rigor and reproducibility of research. AI agents that autonomou…
OpenDerisk: An Industrial Framework for AI-Driven SRE, with Design, Implementation, and Case Studies
Peng Di, Faqiang Chen, Xiao Bai +12
The escalating complexity of modern software imposes an unsustainable operational burden on Site Reliability Engineering (SRE) teams, demanding AI-driven automation that can emulat…
Know Thyself? On the Incapability and Implications of AI Self-Recognition
Xiaoyan Bai, Aryan Shrivastava, Ari Holtzman +1
Self-recognition is a crucial metacognitive capability for AI systems, relevant not only for psychological analysis but also for safety, particularly in evaluative scenarios. Motiv…
Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls
Xiaoyan Bai, Itamar Pres, Yuntian Deng +5
Language models are increasingly capable, yet still fail at a seemingly simple task of multi-digit multiplication. In this work, we study why, by reverse-engineering a model that s…
Concept Incongruence: An Exploration of Time and Death in Role Playing
Xiaoyan Bai, Ike Peng, Aditya Singh +1
Consider this prompt "Draw a unicorn with two horns". Should large language models (LLMs) recognize that a unicorn has only one horn by definition and ask users for clarifications,…