16 papers
Beyond Facts: Benchmarking Distributional Reading Comprehension in Large Language Models
Pei-Fu Guo, Ya-An Tsai, Chun-Chia Hsu +6
While most reading comprehension benchmarks for LLMs focus on factual information that can be answered by localizing specific textual evidence, many real-world tasks require unders…
LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs
Pei-Fu Guo, Yun-Da Tsai, Chun-Chia Hsu +6
Evaluating cross-lingual knowledge transfer in large language models is challenging, as correct answers in a target language may arise either from genuine transfer or from prior ex…
An Efficient Algorithm for Thresholding Monte Carlo Tree Search
Shoma Nameki, Atsuyoshi Nakamura, Junpei Komiyama +1
We introduce the Thresholding Monte Carlo Tree Search problem, in which, given a tree and a threshold , a player must answer whether the root node value of $\math…
ScaleRTL: Scaling LLMs with Reasoning Data and Test-Time Compute for Accurate RTL Code Generation
Chenhui Deng, Yun-Da Tsai, Guan-Ting Liu +2
Recent advances in large language models (LLMs) have enabled near-human performance on software coding benchmarks, but their effectiveness in RTL code generation remains limited du…
Multimodal Chip Physical Design Engineer Assistant
Yun-Da Tsai, Chang-Yu Chao, Liang-Yeh Shen +7
Modern chip physical design relies heavily on Electronic Design Automation (EDA) tools, which often struggle to provide interpretable feedback or actionable guidance for improving…
Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification
Nathaniel Pinckney, Chenhui Deng, Chia-Tung Ho +5
We present the Comprehensive Verilog Design Problems (CVDP) benchmark, a new dataset and infrastructure to advance LLM and agent research in hardware design and verification. CVDP…