12 citations · 39 across the 18 of their papers we have counts for
13 papers · 1 filter
Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
Shengming Zhao, Yuchen Shao, Yuheng Huang +4
Retrieval-Augmented Generation (RAG) has emerged as a critical technique for enhancing large language model (LLM) capabilities. However, practitioners face significant challenges w…
LADEV: A Language-Driven Testing and Evaluation Platform for Vision-Language-Action Models in Robotic Manipulation
Zhijie Wang, Zhehua Zhou, Jiayang Song +3
Building on the advancements of Large Language Models (LLMs) and Vision Language Models (VLMs), recent research has introduced Vision-Language-Action (VLA) models as an integrated…
VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation
Zhijie Wang, Zhehua Zhou, Jiayang Song +3
The rapid advancement of generative AI and multi-modal foundation models has shown significant potential in advancing robotic manipulation. Vision-language-action (VLA) models, in…
LeCov: Multi-level Testing Criteria for Large Language Models
Xuan Xie, Jiayang Song, Yuheng Huang +4
Large Language Models (LLMs) are widely used in many different domains, but because of their limited interpretability, there are questions about how trustworthy they are in various…
MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems
Renzhi Wang, Zhehua Zhou, Jiayang Song +3
Cyber-Physical Systems (CPSs) are increasingly prevalent across various industrial and daily-life domains, with applications ranging from robotic operations to autonomous driving.…
AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling
Yuheng Huang, Jiayang Song, Qiang Hu +2
Performance evaluation plays a crucial role in the development life cycle of large language models (LLMs). It estimates the model's capability, elucidates behavior characteristics,…
Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture
Jiayang Song, Yuheng Huang, Zhehua Zhou +1
As safety remains a crucial concern throughout the development lifecycle of Large Language Models (LLMs), researchers and industrial practitioners have increasingly focused on safe…
Are LLMs Correctly Integrated into Software Systems?
Yuchen Shao, Yuheng Huang, Jiawei Shen +3
Large language models (LLMs) provide effective solutions in various application scenarios, with the support of retrieval-augmented generation (RAG). However, developers face challe…
Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Models
Zhijie Wang, Zijie Zhou, Da Song +4
Large Language Models (LLMs) have demonstrated unprecedented capabilities in code generation. However, there remains a limited understanding of code generation errors that LLMs can…
TESTEVAL: Benchmarking Large Language Models for Test Case Generation
Wenhan Wang, Chenyuan Yang, Zhijie Wang +6
Testing plays a crucial role in the software development cycle, enabling the detection of bugs, vulnerabilities, and other undesirable behaviors. To perform software testing, teste…
GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model
Zhehua Zhou, Xuan Xie, Jiayang Song +2
Safe Reinforcement Learning (SRL) aims to realize a safe learning process for Deep Reinforcement Learning (DRL) algorithms by incorporating safety constraints. However, the efficac…
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward
Xuan Xie, Jiayang Song, Zhehua Zhou +3
While Large Language Models (LLMs) have seen widespread applications across numerous fields, their limited interpretability poses concerns regarding their safe operations from mult…
Evaluation and Improvement of Fault Detection for Large Language Models
Qiang Hu, Jin Wen, Maxime Cordy +4
Large language models (LLMs) have recently achieved significant success across various application domains, garnering substantial attention from different communities. Unfortunatel…