17 citations · 23 across the 5 of their papers we have counts for
3 papers · 1 filter
QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture
Shvetank Prakash, Andrew Cheng, Mark Mazumder +24
The field of computer architecture, which bridges high-level software abstractions and low-level hardware implementations, remains absent from current large language model (LLM) ev…
A2Perf: Real-World Autonomous Agents Benchmark
Ikechukwu Uchendu, Jason Jabbour, Korneel Van den Berghe +15
Autonomous agents and systems cover a number of application areas, from robotics and digital assistants to combinatorial optimization, all sharing common, unresolved research chall…
QuArch: A Question-Answering Dataset for AI Agents in Computer Architecture
Shvetank Prakash, Andrew Cheng, Jason Yik +14
We introduce QuArch, a dataset of 1500 human-validated question-answer pairs designed to evaluate and enhance language models' understanding of computer architecture. The dataset c…