Publications (14)
DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction
Zhijie Wang, Yuheng Huang, Da Song +2
Recurrent Neural Networks (RNNs) have been widely used in Natural Language Processing (NLP) tasks given its superior performance on processing sequential data. However, it is chall…
TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models
Ruoyu Sun, Da Song, Jiayang Song +2
As Large Language Models (LLMs) continue to revolutionize Natural Language Processing (NLP) applications, critical concerns about their trustworthiness persist, particularly in saf…
A huge-amplitude white-light superflare on a L0 brown dwarf discovered by GWAC survey
Li-Ping Xin, Hua-li Li, Jing Wang +21
White-light superflares from ultra cool stars are thought to be resulted from magnetic reconnection, but the magnetic dynamics in a fully convective star is not clear yet. In this…
DeepLens: Interactive Out-of-distribution Data Detection in NLP Models
Da Song, Zhijie Wang, Yuheng Huang +2
Machine Learning (ML) has been widely used in Natural Language Processing (NLP) applications. A fundamental assumption in ML is that training data and real-world data should follow…
Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Models
Zhijie Wang, Zijie Zhou, Da Song +4
Large Language Models (LLMs) have demonstrated unprecedented capabilities in code generation. However, there remains a limited understanding of code generation errors that LLMs can…
Evaluating Implicit Regulatory Compliance in LLM Tool Invocation via Logic-Guided Synthesis
Da Song, Yuheng Huang, Boqi Chen +4
The integration of large language models (LLMs) into autonomous agents has enabled complex tool use, yet in high-stakes domains, these systems must strictly adhere to regulatory st…
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
Zhijie Wang, Yuheng Huang, Da Song +2
The recent advancements in Generative AI have significantly advanced the field of text-to-image generation. The state-of-the-art text-to-image model, Stable Diffusion, is now capab…
LeCov: Multi-level Testing Criteria for Large Language Models
Xuan Xie, Jiayang Song, Yuheng Huang +4
Large Language Models (LLMs) are widely used in many different domains, but because of their limited interpretability, there are questions about how trustworthy they are in various…
LUNA: A Model-Based Universal Analysis Framework for Large Language Models
Da Song, Xuan Xie, Jiayang Song +4
Over the past decade, Artificial Intelligence (AI) has had great success recently and is being used in a wide range of academic and industrial fields. More recently, LLMs have made…
Evaluating LLMs on Sequential API Call Through Automated Test Generation
Yuheng Huang, Jiayang Song, Da Song +4
By integrating tools from external APIs, Large Language Models (LLMs) have expanded their promising capabilities in a diverse spectrum of complex real-world tasks. However, testing…
TESTEVAL: Benchmarking Large Language Models for Test Case Generation
Wenhan Wang, Chenyuan Yang, Zhijie Wang +6
Testing plays a crucial role in the software development cycle, enabling the detection of bugs, vulnerabilities, and other undesirable behaviors. To perform software testing, teste…
An Image Simulator of Lunar Far-Side Impact Flashes Captured from the Earth-Moon L2 Point
Da Song, Hong-bo Cai, Shen Wang +1
Impact flashes on the moon are caused by high-speed collisions of celestial bodies with the lunar surface. The study of the impacts is critical for exploring the evolutionary histo…
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward
Xuan Xie, Jiayang Song, Zhehua Zhou +3
While Large Language Models (LLMs) have seen widespread applications across numerous fields, their limited interpretability poses concerns regarding their safe operations from mult…
Structural Anchors and Reasoning Fragility:Understanding CoT Robustness in LLM4Code
Yang Liu, Da Song, Armstrong Foundjem +2
Chain-of-Thought (CoT) prompting is widely used to elicit explicit reasoning from large language models for code (LLM4Code). However, its impact on robustness and the stability of…