77 citations · 208 across the 15 of their papers we have counts for
16 papers
The Wording Effect: Quantifying Two-Way Drift in LLM Benchmark Performance
Shailja Thakur, Sungeun An, Chad DeLuca +1
A benchmark score comes from a single phrasing of each problem. That single phrasing is treated as if it stood for the whole space of ways the same problem could be asked, but it d…
A Systematic Approach for Large Language Models Debugging
Basel Shbita, Anna Lisa Gentile, Bing Zhang +10
Large language models (LLMs) have become central to modern AI workflows, powering applications from open-ended text generation to complex agent-based reasoning. However, debugging…
STaD: Scaffolded Task Design for Identifying Compositional Skill Gaps in LLMs
Sungeun An, Swanand Ravindra Kadhe, Shailja Thakur +2
Benchmarks are often used as a standard to understand LLM capabilities in different domains. However, aggregate benchmark scores provide limited insight into compositional skill ga…
Generating Verifiable Chain of Thoughts from Exection-Traces
Shailja Thakur, Vaibhav Saxena, Rohan Kulkarni +4
Getting language models to reason correctly about code requires training on data where each reasoning step can be checked. Current synthetic Chain-of-Thought (CoT) training data of…
Automatically Improving LLM-based Verilog Generation using EDA Tool Feedback
Jason Blocklove, Shailja Thakur, Benjamin Tan +3
Traditionally, digital hardware designs are written in the Verilog hardware description language (HDL) and debugged manually by engineers. This can be time-consuming and error-pron…
Make Every Move Count: LLM-based High-Quality RTL Code Generation Using MCTS
Matthew DeLorenzo, Animesh Basak Chowdhury, Vasudev Gohil +4
Existing large language models (LLMs) for register transfer level code generation face challenges like compilation failures and suboptimal power, performance, and area (PPA) effici…
AutoChip: Automating HDL Generation Using LLM Feedback
Shailja Thakur, Jason Blocklove, Hammond Pearce +3
Traditionally, designs are written in Verilog hardware description language (HDL) and debugged by hardware engineers. While this approach is effective, it is time-consuming and err…
Towards the Imagenets of ML4EDA
Animesh Basak Chowdhury, Shailja Thakur, Hammond Pearce +2
Despite the growing interest in ML-guided EDA tools from RTL to GDSII, there are no standard datasets or prototypical learning tasks defined for the EDA problem domain. Experience…
Are Emily and Greg Still More Employable than Lakisha and Jamal? Investigating Algorithmic Hiring Bias in the Era of ChatGPT
Akshaj Kumar Veldanda, Fabian Grob, Shailja Thakur +4
Large Language Models (LLMs) such as GPT-3.5, Bard, and Claude exhibit applicability across numerous tasks. One domain of interest is their use in algorithmic hiring, specifically…
VeriGen: A Large Language Model for Verilog Code Generation
Shailja Thakur, Baleegh Ahmad, Hammond Pearce +4
In this study, we explore the capability of Large Language Models (LLMs) to automate hardware design by generating high-quality Verilog code, a common language for designing and mo…
(Security) Assertions by Large Language Models
Rahul Kande, Hammond Pearce, Benjamin Tan +4
The security of computer systems typically relies on a hardware root of trust. As vulnerabilities in hardware can have severe implications on a system, there is a need for techniqu…
Fixing Hardware Security Bugs with Large Language Models
Baleegh Ahmad, Shailja Thakur, Benjamin Tan +2
Novel AI-based code-writing Large Language Models (LLMs) such as OpenAI's Codex have demonstrated capabilities in many coding-adjacent domains. In this work we consider how LLMs ma…
Security and Interpretability in Automotive Systems
Shailja Thakur
The lack of any sender authentication mechanism in place makes CAN (Controller Area Network) vulnerable to security threats. For instance, an attacker can impersonate an ECU (Elect…
Benchmarking Large Language Models for Automated Verilog RTL Code Generation
Shailja Thakur, Baleegh Ahmad, Zhenxing Fan +5
Automating hardware design could obviate a significant amount of human error from the engineering process and lead to fewer errors. Verilog is a popular hardware description langua…
A generalizable saliency map-based interpretation of model outcome
Shailja Thakur, Sebastian Fischmeister
One of the significant challenges of deep neural networks is that the complex nature of the network prevents human comprehension of the outcome of the network. Consequently, the ap…
CANOA: CAN Origin Authentication Through Power Side-Channel Monitoring
Shailja Thakur, Carlos Moreno, Sebastian Fischmeister
The lack of any sender authentication mechanism in place makes CAN (Controller Area Network) vulnerable to security threats. For instance, an attacker can impersonate an ECU (Elect…