9 papers
Can Machines Really See Objects in Images? A Study Based on Syntactic Distance and Visual Self-Referential Instances
Xingyu Peng, Junran Wu, Yue Hou +9
Can a vision model truly see an object, or does it only fit surface-level visual cues? Following Wittgenstein's view that the limits of language are the limits of the world, we vie…
NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
Jingzhe Ding, Shengda Long, Changxin Pu +46
Recent advances in coding agents suggest rapid progress toward autonomous software development, yet existing benchmarks fail to rigorously evaluate the long-horizon capabilities re…
Graph Out-of-Distribution Detection via Test-Time Calibration with Dual Dynamic Dictionaries
Yue Hou, Ruomei Liu, Yingke Su +2
A key challenge in graph out-of-distribution (OOD) detection lies in the absence of ground-truth OOD samples during training. Existing methods are typically optimized to capture fe…
Redundancy-Aware Test-Time Graph Out-of-Distribution Detection
Yue Hou, He Zhu, Ruomei Liu +3
Distributional discrepancy between training and test data can lead models to make inaccurate predictions when encountering out-of-distribution (OOD) samples in real-world applicati…
Auto-ARGUE: LLM-Based Report Generation Evaluation
William Walden, Marc Mason, Orion Weller +10
Generation of citation-backed reports is a primary use case for retrieval-augmented generation (RAG) systems. While open-source evaluation tools exist for various RAG tasks, tools…
Efficient Agents: Building Effective Agents While Reducing Cost
Ningning Wang, Xavier Hu, Pai Liu +11
The remarkable capabilities of Large Language Model (LLM)-driven agents have enabled sophisticated systems to tackle complex, multi-step tasks, but their escalating costs threaten…
Structural Entropy Guided Unsupervised Graph Out-Of-Distribution Detection
Yue Hou, He Zhu, Ruomei Liu +4
With the emerging of huge amount of unlabeled data, unsupervised out-of-distribution (OOD) detection is vital for ensuring the reliability of graph neural networks (GNNs) by identi…
NC-NCD: Novel Class Discovery for Node Classification
Yue Hou, Xueyuan Chen, He Zhu +5
Novel Class Discovery (NCD) involves identifying new categories within unlabeled data by utilizing knowledge acquired from previously established categories. However, existing NCD…
HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification
He Zhu, Junran Wu, Ruomei Liu +5
Existing self-supervised methods in natural language processing (NLP), especially hierarchical text classification (HTC), mainly focus on self-supervised contrastive learning, extr…