1 citations · 1 across the 14 of their papers we have counts for
15 papers
LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers
Jingyu Hu, Shu Yang, Weiru Liu +1
Chain-of-Thought (CoT) reasoning has been shown to improve the performance of large language models (LLMs), yet existing optimization methods largely rely on outcome-based feedback…
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
Xiangyi Li, Yimin Liu, Wenbo Chen +75
Agent Skills are structured packages of procedural knowledge that augment large language model (LLM) agents at inference time. Despite rapid adoption, there is no standard way to m…
Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Models
Xilin Gong, Shu Yang, Zehua Cao +2
Large Language Models (LLMs) have demonstrated strong capabilities for hidden representation interpretation through Patchscopes, a framework that uses LLMs themselves to generate h…
Hearing is Believing? Evaluating and Analyzing Audio Language Model Sycophancy with SYAUDIO
Junchi Yao, Lokranjan Lakshmikanthan, Annie Zhao +5
Audio Language Models (ALMs) have recently shown strong capabilities in unified reasoning over speech, sound, and natural language; yet we find that they can inherit sycophancy, th…
Not All Code Is Equal: A Data-Centric Study of Code Complexity and LLM Reasoning
Lukas Twist, Shu Yang, Hanqi Yan +4
Large Language Models (LLMs) increasingly exhibit strong reasoning abilities, often attributed to their capacity to generate chain-of-thought-style intermediate reasoning. Recent w…
Dependently-Typed AARA: A Non-Affine Approach for Resource Analysis of Higher-Order Programs
Han Xu, Di Wang
Static resource analysis determines the resource consumption (e.g., time complexity) of a program without executing it. Among the numerous existing approaches for resource analysis…
A Generalizable Framework for Building Executable Domain-Specific LLMs under Data Scarcity: Demonstration on Semiconductor TCAD Simulation
Di Wang, Zhenhua Wu, Yu Liu +2
Scientific and engineering verticals often suffer from data scarcity and strict executability requirements: models must generate not only fluent text, but also syntactically valid,…
AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor
Shu Yang, Jingyu Hu, Tong Li +3
We introduce AutoMonitor-Bench, the first benchmark designed to systematically evaluate the reliability of LLM-based misbehavior monitors across diverse tasks and failure modes. Au…
AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent
Haipeng Luo, Huawen Feng, Qingfeng Sun +6
Large Reasoning Models (LRMs) like o3 and DeepSeek-R1 have achieved remarkable progress in reasoning tasks with long cot. However, they remain computationally inefficient and strug…
MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models
Jingyu Hu, Shu Yang, Xilin Gong +3
Large Reasoning Models (LRMs) suffer from sycophantic behavior, where models tend to agree with users' incorrect beliefs and follow misinformation rather than maintain independent…
Investigating CoT Monitorability in Large Reasoning Models
Shu Yang, Junchao Wu, Xilin Gong +4
Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex tasks by engaging in extended reasoning before producing final answers. Beyond improving abilities…
PathFuzzing: Worst Case Analysis by Fuzzing Symbolic-Execution Paths
Zimu Chen, Di Wang
Estimating worst-case resource consumption is a critical task in software development. The worst-case analysis (WCA) problem is an optimization-based abstraction of this task. Fuzz…
A Program Logic for Under-approximating Worst-case Resource Usage
Ziyue Jin, Di Wang
Understanding and predicting the worst-case resource usage is crucial for software quality; however, existing methods either over-approximate with potentially loose bounds or under…
Benchmarking Language Model Creativity: A Case Study on Code Generation
Yining Lu, Dixuan Wang, Tianjian Li +4
As LLMs become increasingly prevalent, it is interesting to consider how ``creative'' these models can be. From cognitive science, creativity consists of at least two key character…
Worst-Case Analysis is Maximum-A-Posteriori Estimation
Hongjun Wu, Di Wang
The worst-case resource usage of a program can provide useful information for many software-engineering tasks, such as performance optimization and algorithmic-complexity-vulnerabi…