106 citations · 192 across the 24 of their papers we have counts for
24 papers
NARU: A Benchmark for NARrative Evolution and Cultural Nuance Understanding in Japanese Extreme Long Video
Yuheng Huang, Jianlang Chen, Jiayang Song +6
Long-form video understanding encompasses tasks that go beyond retrieving isolated events, including tracking an evolving narrative and interpreting social meaning that may remain…
Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphs
Xiaoning Ren, Yinxing Xue, Lei Ma +1
As Code Large Language Models (LLMs) become central to modern software engineering, their inherent stochasticity poses significant real-world risks, where even minor errors can lea…
Evaluating Implicit Regulatory Compliance in LLM Tool Invocation via Logic-Guided Synthesis
Da Song, Yuheng Huang, Boqi Chen +4
The integration of large language models (LLMs) into autonomous agents has enabled complex tool use, yet in high-stakes domains, these systems must strictly adhere to regulatory st…
TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models
Ruoyu Sun, Da Song, Jiayang Song +2
As Large Language Models (LLMs) continue to revolutionize Natural Language Processing (NLP) applications, critical concerns about their trustworthiness persist, particularly in saf…
Evaluating LLMs on Sequential API Call Through Automated Test Generation
Yuheng Huang, Jiayang Song, Da Song +4
By integrating tools from external APIs, Large Language Models (LLMs) have expanded their promising capabilities in a diverse spectrum of complex real-world tasks. However, testing…
Risk Assessment Framework for Code LLMs via Leveraging Internal States
Yuheng Huang, Lei Ma, Keizaburo Nishikino +1
The pre-training paradigm plays a key role in the success of Large Language Models (LLMs), which have been recognized as one of the most significant advancements of AI recently. Bu…
Foundation Models for Autonomous Driving System: An Initial Roadmap
Xiongfei Wu, Mingfei Cheng, Xiaoning Ren +8
Recent advances in foundation models (FMs), including large language models (LLMs), vision-language models (VLMs), and world models, have opened new opportunities for autonomous dr…
Fine-grained Testing for Autonomous Driving Software: a Study on Autoware with LLM-driven Unit Testing
Wenhan Wang, Xuan Xie, Yuheng Huang +3
Testing autonomous driving systems (ADS) is critical to ensuring their reliability and safety. Existing ADS testing works focuses on designing scenarios to evaluate system-level be…
Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
Shengming Zhao, Yuchen Shao, Yuheng Huang +4
Retrieval-Augmented Generation (RAG) has emerged as a critical technique for enhancing large language model (LLM) capabilities. However, practitioners face significant challenges w…
LADEV: A Language-Driven Testing and Evaluation Platform for Vision-Language-Action Models in Robotic Manipulation
Zhijie Wang, Zhehua Zhou, Jiayang Song +3
Building on the advancements of Large Language Models (LLMs) and Vision Language Models (VLMs), recent research has introduced Vision-Language-Action (VLA) models as an integrated…
VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation
Zhijie Wang, Zhehua Zhou, Jiayang Song +3
The rapid advancement of generative AI and multi-modal foundation models has shown significant potential in advancing robotic manipulation. Vision-language-action (VLA) models, in…
LeCov: Multi-level Testing Criteria for Large Language Models
Xuan Xie, Jiayang Song, Yuheng Huang +4
Large Language Models (LLMs) are widely used in many different domains, but because of their limited interpretability, there are questions about how trustworthy they are in various…
AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling
Yuheng Huang, Jiayang Song, Qiang Hu +2
Performance evaluation plays a crucial role in the development life cycle of large language models (LLMs). It estimates the model's capability, elucidates behavior characteristics,…
Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture
Jiayang Song, Yuheng Huang, Zhehua Zhou +1
As safety remains a crucial concern throughout the development lifecycle of Large Language Models (LLMs), researchers and industrial practitioners have increasingly focused on safe…
Are LLMs Correctly Integrated into Software Systems?
Yuchen Shao, Yuheng Huang, Jiawei Shen +3
Large language models (LLMs) provide effective solutions in various application scenarios, with the support of retrieval-augmented generation (RAG). However, developers face challe…
Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Models
Zhijie Wang, Zijie Zhou, Da Song +4
Large Language Models (LLMs) have demonstrated unprecedented capabilities in code generation. However, there remains a limited understanding of code generation errors that LLMs can…
TESTEVAL: Benchmarking Large Language Models for Test Case Generation
Wenhan Wang, Chenyuan Yang, Zhijie Wang +6
Testing plays a crucial role in the software development cycle, enabling the detection of bugs, vulnerabilities, and other undesirable behaviors. To perform software testing, teste…
GMP-TL: Gender-augmented Multi-scale Pseudo-label Enhanced Transfer Learning for Speech Emotion Recognition
Yu Pan, Yuguang Yang, Heng Lu +2
The continuous evolution of pre-trained speech models has greatly advanced Speech Emotion Recognition (SER). However, current research typically relies on utterance-level emotion l…
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward
Xuan Xie, Jiayang Song, Zhehua Zhou +3
While Large Language Models (LLMs) have seen widespread applications across numerous fields, their limited interpretability poses concerns regarding their safe operations from mult…
Evaluation and Improvement of Fault Detection for Large Language Models
Qiang Hu, Jin Wen, Maxime Cordy +4
Large language models (LLMs) have recently achieved significant success across various application domains, garnering substantial attention from different communities. Unfortunatel…
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
Zhijie Wang, Yuheng Huang, Da Song +2
The recent advancements in Generative AI have significantly advanced the field of text-to-image generation. The state-of-the-art text-to-image model, Stable Diffusion, is now capab…
LUNA: A Model-Based Universal Analysis Framework for Large Language Models
Da Song, Xuan Xie, Jiayang Song +4
Over the past decade, Artificial Intelligence (AI) has had great success recently and is being used in a wide range of academic and industrial fields. More recently, LLMs have made…
MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition
Yu Pan, Yuguang Yang, Yuheng Huang +6
Despite notable progress, speech emotion recognition (SER) remains challenging due to the intricate and ambiguous nature of speech emotion, particularly in wild world. While curren…
Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models
Yuheng Huang, Jiayang Song, Zhijie Wang +4
The recent performance leap of Large Language Models (LLMs) opens up new opportunities across numerous industrial applications and domains. However, erroneous generations, such as…