18 papers
From Copying to Composition: How Language Models Develop In-Context Learning
Minglai Yang, Xinyu Guo, Renxiong Wang +3
In-context learning (ICL) allows language models to adapt their behavior from instructions and demonstrations in a prompt, but accuracy alone does not reveal how contextual informa…
CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat
Bangya Liu, Jianhong Tu, Yueyang Pan +18
AI agents are demonstrating increasingly strong offensive and defensive cybersecurity capabilities in the real world. Current benchmarks mostly evaluate agents in isolation against…
Evidence-First Agentic Verification
Reza Sanayei, Razvan-Gabriel Dumitru, Sunjie Hou +7
Long-horizon, semi-verifiable agentic tasks often rely on rubric-based evaluation, but the evidence needed to assess these tasks can be spread across the environment state and the…
Structural properties of admissible subgraphs and the depth of symbolic powers of cover ideals
Nguyen Thu Hang, Nguyen Thi Thanh Tam, Thanh Vu
Let be a simple graph with cover ideal in a polynomial ring $S = \k[x_1, \ldots, x_n]$. For each integer , let denote the maximum size of an ordered ma…
READY or Not: Reliable Enterprise Agent Deployment
Veronica Chatrath, Bryan Zhu, Jingxuan Fan +15
An AI agent can perform well on benchmarks and still be unsuitable for deployment. Existing AI-agent benchmarks measure whether an agent can complete realistic professional work, w…
Spine-Branch Coordination for Multi-agent Computer Use
Mian Zhang, Manasi Sharma, Sheng Zhang +5
Computer use agents (CUAs) are increasingly deployed as multi-agent systems that decompose a task into multiple subtasks executed across parallel virtual machines (VMs). However, a…
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-Based RL
Minglai Yang, Xinyu Guo, Zhengliang Shi +8
Rubric-based reinforcement learning, in which an LLM judge scores responses against prompt-specific criteria, has become a standard way to post-train language models on tasks witho…
Knowledge Index of Noah's Ark
Sheng Jin, Minghao Liu, Yunze Xiao +24
Knowledge benchmarks for LLMs face three issues: scaling-driven designs that do not operationalize disciplinary representativeness; flat-payment annotation that permits lazy consen…
Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
Minglai Yang, Xinyan Velocity Yu, Pengyuan Li +22
Document parsing and recognition are fundamental capabilities for vision-language models (VLMs) and document processing systems. However, existing Optical Character Recognition (OC…
EchoRL: Reinforcement Learning via Rollout Echoing
Jinhe Bi, Aniri, Minglai Yang +9
Reinforcement Learning with Verifiable Rewards is an effective route for post-training to strengthen the reasoning capability of large language models. However, as training proceed…
Triaging Threats to Specialized Guardrails
Wenjie Jacky Mo, Xiaofei Wen, Rui Cai +6
Building robust safety guardrails is essential for deploying Large Language Models across diverse real-world applications. However, this goal remains challenging because safety ris…
Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality
Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato +8
As LLMs are deployed in high-stakes settings, users must judge the correctness of individual responses, often relying on model-generated justifications such as reasoning chains or…
AlignSAE: Concept-Aligned Sparse Autoencoders
Minglai Yang, Xinyu Guo, Zhengliang Shi +4
Large Language Models (LLMs) encode factual knowledge within hidden parametric spaces that are difficult to inspect or control. While Sparse Autoencoders (SAEs) can decompose hidde…
The Answer Lies Within: Self-Derived Rewards Enable Explainable Relation Extraction
Xinyu Guo, Zhengliang Shi, Minglai Yang +2
Despite the remarkable reasoning capabilities of large language models, they still struggle with one-shot relation extraction without predefined relation labels. We identify two pi…
Word2VecGD: Neural Graph Drawing with Cosine-Stress Optimization
Minglai Yang, Reyan Ahmed
We propose a novel graph visualization method leveraging random walk-based embeddings to replace costly graph-theoretical distance computations. Using word2vec-inspired embeddings,…
Improving the Data-efficiency of Reinforcement Learning by Warm-starting with LLM
Thang Duong, Minglai Yang, Chicheng Zhang
We investigate the usage of Large Language Model (LLM) in collecting high-quality data to warm-start Reinforcement Learning (RL) algorithms for learning in some classical Markov De…
How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark
Minglai Yang, Ethan Huang, Liang Zhang +3
We introduce Grade School Math with Distracting Context (GSM-DC), a synthetic benchmark to evaluate Large Language Models' (LLMs) reasoning robustness against systematically contro…
CopySpec: Accelerating LLMs with Speculative Copy-and-Paste Without Compromising Quality
Razvan-Gabriel Dumitru, Minglai Yang, Vikas Yadav +1
We introduce CopySpec, a simple yet effective technique to tackle the inefficiencies LLMs face when generating responses that closely resemble previous outputs or responses that ca…