5 papers
SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
Jeffrey Jian Ma, Milad Hashemi, Amir Yazdanbakhsh +5
Optimizing the performance of large-scale software repositories demands expertise in code reasoning and software engineering (SWE) to reduce runtime while preserving program correc…
JAXBench: Benchmarking Autonomous TPU Kernel Optimization
Arya Tschand, Charles Hong, Julian Walker +7
Rigorous benchmarks have driven progress in autonomous GPU kernel performance optimization by establishing a shared target to hillclimb on, but no equivalent exists for TPUs. We pr…
QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture
Shvetank Prakash, Andrew Cheng, Arya Tschand +25
The field of computer architecture, which bridges high-level software abstractions and low-level hardware implementations, remains absent from current large language model (LLM) ev…
COSMIC: Enabling Full-Stack Co-Design and Optimization of Distributed Machine Learning Systems
Aditi Raju, Jared Ni, William Won +6
Large-scale machine learning models necessitate distributed systems, posing significant design challenges due to the large parameter space across distinct design stacks. Existing s…
QuArch: A Question-Answering Dataset for AI Agents in Computer Architecture
Shvetank Prakash, Andrew Cheng, Jason Yik +14
We introduce QuArch, a dataset of 1500 human-validated question-answer pairs designed to evaluate and enhance language models' understanding of computer architecture. The dataset c…