2 papers
cs.SE2026
IDE-Bench: Evaluating Large Language Models as IDE Agents on Real-World Software Engineering Tasks
Spencer Mateega, Jeff Yang, Tiana Costello +3
IDE-Bench is a comprehensive framework for evaluating AI IDE agents on real-world software engineering tasks through an IDE-native tool interface. We present a Dockerized test harn…
cs.CL2025
UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Tools
Sam Jung, Agustin Garcinuno, Spencer Mateega
AI text-to-app tools promise high quality applications and websites in minutes, yet no public benchmark rigorously verifies those claims. We introduce UI-Bench, the first large-sca…