48 citations · 160 across the 22 of their papers we have counts for
13 papers · 1 filter
Unraveling the Efficacy of Neuron Coverage Criteria on Testing Large Language Models
Xiaoning Ren, Jiayang Song, Chongyang Liu +3
Neuron coverage criteria aim to assess test suite quality and guide test input generation, but their effectiveness for Transformer-based language models remains unclear. We evaluat…
TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models
Ruoyu Sun, Da Song, Jiayang Song +2
As Large Language Models (LLMs) continue to revolutionize Natural Language Processing (NLP) applications, critical concerns about their trustworthiness persist, particularly in saf…
Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
Shengming Zhao, Yuchen Shao, Yuheng Huang +4
Retrieval-Augmented Generation (RAG) has emerged as a critical technique for enhancing large language model (LLM) capabilities. However, practitioners face significant challenges w…
VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation
Zhijie Wang, Zhehua Zhou, Jiayang Song +3
The rapid advancement of generative AI and multi-modal foundation models has shown significant potential in advancing robotic manipulation. Vision-language-action (VLA) models, in…
LeCov: Multi-level Testing Criteria for Large Language Models
Xuan Xie, Jiayang Song, Yuheng Huang +4
Large Language Models (LLMs) are widely used in many different domains, but because of their limited interpretability, there are questions about how trustworthy they are in various…
MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems
Renzhi Wang, Zhehua Zhou, Jiayang Song +3
Cyber-Physical Systems (CPSs) are increasingly prevalent across various industrial and daily-life domains, with applications ranging from robotic operations to autonomous driving.…
AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling
Yuheng Huang, Jiayang Song, Qiang Hu +2
Performance evaluation plays a crucial role in the development life cycle of large language models (LLMs). It estimates the model's capability, elucidates behavior characteristics,…
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward
Xuan Xie, Jiayang Song, Zhehua Zhou +3
While Large Language Models (LLMs) have seen widespread applications across numerous fields, their limited interpretability poses concerns regarding their safe operations from mult…
Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation
Zhehua Zhou, Jiayang Song, Xuan Xie +5
As a representative cyber-physical system (CPS), robotic manipulator has been widely adopted in various academic research and industrial processes, indicating its potential to act…
Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models
Yuheng Huang, Jiayang Song, Zhijie Wang +4
The recent performance leap of Large Language Models (LLMs) opens up new opportunities across numerous industrial applications and domains. However, erroneous generations, such as…
Mosaic: Model-based Safety Analysis Framework for AI-enabled Cyber-Physical Systems
Xuan Xie, Jiayang Song, Zhehua Zhou +2
Cyber-physical systems (CPSs) are now widely deployed in many industrial domains, e.g., manufacturing systems and autonomous vehicles. To further enhance the capability and applica…
AutoRepair: Automated Repair for AI-Enabled Cyber-Physical Systems under Safety-Critical Conditions
Deyun Lyu, Jiayang Song, Zhenya Zhang +4
Cyber-Physical Systems (CPS) have been widely deployed in safety-critical domains such as transportation, power and energy. Recently, there comes an increasing demand in employing…
When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward
Jiayang Song, Deyun Lyu, Zhenya Zhang +3
Cyber-physical systems (CPS) have been broadly deployed in safety-critical domains, such as automotive systems, avionics, medical devices, etc. In recent years, Artificial Intellig…