8 papers
When LLMs Over-Answer: Measuring and Mitigating Quality Issues in LLM-Based Hardware Description Language Question Answering
Ziteng Hu, Jiachi Chen, Wenhao Lv +2
The rapid advancement of large language models (LLMs) has led practitioners to increasingly rely on them for answering questions about hardware description languages (HDLs). Becaus…
FORTIS: Benchmarking Over-Privilege in Agent Skills
Shawn Li, Chenxiao Yu, Han Wang +8
Large language model agents increasingly operate through an intermediate skill layer that mediates between user intent and concrete task execution. This layer is widely treated as…
Improving Code Translation with Syntax-Guided and Semantic-aware Preference Optimization
Yuhan Wu, Huan Zhang, Wei Cheng +3
LLMs have shown immense potential for code translation, yet they often struggle to ensure both syntactic correctness and semantic consistency. While preference-based learning offer…
CoT-Guard: Small Models for Strong Monitoring
Nirav Diwan, Han Wang, Berkcan Kapusuzoglu +6
Monitoring the chain-of-thought (CoT) of reasoning models is a promising approach for detecting covert misbehavior (i.e., hidden objectives) in code generation tasks. While large m…
Bootstrapping Code Translation with Weighted Multilanguage Exploration
Yuhan Wu, Huan Zhang, Wei Cheng +3
Code translation across multiple programming languages is essential yet challenging due to two vital obstacles: scarcity of parallel data paired with executable test oracles, and o…
MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
Han Wang, Yifan Sun, Brian Ko +8
Large language models (LLMs) can generate chains of thought (CoTs) that are not always causally responsible for their final outputs. When such a mismatch occurs, the CoT no longer…