48 citations · 160 across the 22 of their papers we have counts for
22 papers
Unraveling the Efficacy of Neuron Coverage Criteria on Testing Large Language Models
Xiaoning Ren, Jiayang Song, Chongyang Liu +3
Neuron coverage criteria aim to assess test suite quality and guide test input generation, but their effectiveness for Transformer-based language models remains unclear. We evaluat…
NARU: A Benchmark for NARrative Evolution and Cultural Nuance Understanding in Japanese Extreme Long Video
Yuheng Huang, Jianlang Chen, Jiayang Song +6
Long-form video understanding encompasses tasks that go beyond retrieving isolated events, including tracking an evolving narrative and interpreting social meaning that may remain…
MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Models
Xincheng Yao, Zefeng Qian, Chao Shi +2
In the progress of industrial anomaly detection, general anomaly detection (GAD) is an emerging trend and also the ultimate goal. Unlike the conventional single- and multi-class AD…
TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models
Ruoyu Sun, Da Song, Jiayang Song +2
As Large Language Models (LLMs) continue to revolutionize Natural Language Processing (NLP) applications, critical concerns about their trustworthiness persist, particularly in saf…
Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
Shengming Zhao, Yuchen Shao, Yuheng Huang +4
Retrieval-Augmented Generation (RAG) has emerged as a critical technique for enhancing large language model (LLM) capabilities. However, practitioners face significant challenges w…
LADEV: A Language-Driven Testing and Evaluation Platform for Vision-Language-Action Models in Robotic Manipulation
Zhijie Wang, Zhehua Zhou, Jiayang Song +3
Building on the advancements of Large Language Models (LLMs) and Vision Language Models (VLMs), recent research has introduced Vision-Language-Action (VLA) models as an integrated…
VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation
Zhijie Wang, Zhehua Zhou, Jiayang Song +3
The rapid advancement of generative AI and multi-modal foundation models has shown significant potential in advancing robotic manipulation. Vision-language-action (VLA) models, in…
LeCov: Multi-level Testing Criteria for Large Language Models
Xuan Xie, Jiayang Song, Yuheng Huang +4
Large Language Models (LLMs) are widely used in many different domains, but because of their limited interpretability, there are questions about how trustworthy they are in various…
MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems
Renzhi Wang, Zhehua Zhou, Jiayang Song +3
Cyber-Physical Systems (CPSs) are increasingly prevalent across various industrial and daily-life domains, with applications ranging from robotic operations to autonomous driving.…
AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling
Yuheng Huang, Jiayang Song, Qiang Hu +2
Performance evaluation plays a crucial role in the development life cycle of large language models (LLMs). It estimates the model's capability, elucidates behavior characteristics,…
Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture
Jiayang Song, Yuheng Huang, Zhehua Zhou +1
As safety remains a crucial concern throughout the development lifecycle of Large Language Models (LLMs), researchers and industrial practitioners have increasingly focused on safe…
GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model
Zhehua Zhou, Xuan Xie, Jiayang Song +2
Safe Reinforcement Learning (SRL) aims to realize a safe learning process for Deep Reinforcement Learning (DRL) algorithms by incorporating safety constraints. However, the efficac…
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward
Xuan Xie, Jiayang Song, Zhehua Zhou +3
While Large Language Models (LLMs) have seen widespread applications across numerous fields, their limited interpretability poses concerns regarding their safe operations from mult…
Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in Robotics
Jiayang Song, Zhehua Zhou, Jiawei Liu +3
Although Deep Reinforcement Learning (DRL) has achieved notable success in numerous robotic applications, designing a high-performing reward function remains a challenging task tha…
LUNA: A Model-Based Universal Analysis Framework for Large Language Models
Da Song, Xuan Xie, Jiayang Song +4
Over the past decade, Artificial Intelligence (AI) has had great success recently and is being used in a wide range of academic and industrial fields. More recently, LLMs have made…
ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning
Zhehua Zhou, Jiayang Song, Kunpeng Yao +2
Motivated by the substantial achievements observed in Large Language Models (LLMs) in the field of natural language processing, recent research has commenced investigations into th…
Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation
Zhehua Zhou, Jiayang Song, Xuan Xie +5
As a representative cyber-physical system (CPS), robotic manipulator has been widely adopted in various academic research and industrial processes, indicating its potential to act…
Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models
Yuheng Huang, Jiayang Song, Zhijie Wang +4
The recent performance leap of Large Language Models (LLMs) opens up new opportunities across numerous industrial applications and domains. However, erroneous generations, such as…
Mosaic: Model-based Safety Analysis Framework for AI-enabled Cyber-Physical Systems
Xuan Xie, Jiayang Song, Zhehua Zhou +2
Cyber-physical systems (CPSs) are now widely deployed in many industrial domains, e.g., manufacturing systems and autonomous vehicles. To further enhance the capability and applica…
AutoRepair: Automated Repair for AI-Enabled Cyber-Physical Systems under Safety-Critical Conditions
Deyun Lyu, Jiayang Song, Zhenya Zhang +4
Cyber-Physical Systems (CPS) have been widely deployed in safety-critical domains such as transportation, power and energy. Recently, there comes an increasing demand in employing…
When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward
Jiayang Song, Deyun Lyu, Zhenya Zhang +3
Cyber-physical systems (CPS) have been broadly deployed in safety-critical domains, such as automotive systems, avionics, medical devices, etc. In recent years, Artificial Intellig…
A Performance Analysis Model of TCP over Multiple Heterogeneous Paths for 5G Mobile Services
Jiayang Song, Ping Dong, Huachun Zhou +3
Driven by the primary requirement of emerging 5G mobile services, the demand for concurrent multipath transfer (CMT) is still prominent. Yet, multipath transport protocols are not…