102 citations · 366 across the 31 of their papers we have counts for
22 papers · 1 filter
AL-Bench: A Benchmark for Automatic Logging
Boyin Tan, Junjielong Xu, Zhouruixing Zhu +1
Logging, the practice of inserting log statements into source code, is critical for improving software reliability. Recently, language model-based techniques have been developed to…
Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12
Code-related benchmarks play a critical role in evaluating large language models (LLMs), yet their quality fundamentally shapes how the community interprets model capabilities. In…
SPES: Towards Optimizing Performance-Resource Trade-Off for Serverless Functions
Cheryl Lee, Zhouruixing Zhu, Tianyi Yang +4
As an emerging cloud computing deployment paradigm, serverless computing is gaining traction due to its efficiency and ability to harness on-demand cloud resources. However, a sign…
Go Static: Contextualized Logging Statement Generation
Yichen Li, Yintong Huo, Renyi Zhong +6
Logging practices have been extensively investigated to assist developers in writing appropriate logging statements for documenting software behaviors. Although numerous automatic…
LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models
Yuxuan Wan, Wenxuan Wang, Yiliu Yang +5
We introduce LogicAsker, a novel approach for evaluating and enhancing the logical reasoning capabilities of large language models (LLMs) such as ChatGPT and GPT-4. Despite LLMs' p…
LILAC: Log Parsing using LLMs with Adaptive Parsing Cache
Zhihan Jiang, Jinyang Liu, Zhuangbin Chen +6
Log parsing transforms log messages into structured formats, serving as the prerequisite step for various log analysis tasks. Although a variety of log parsing approaches have been…
An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software
Wenxuan Wang, Jingyuan Huang, Jen-tse Huang +4
The exponential growth of social media platforms has brought about a revolution in communication and content dissemination in human society. Nevertheless, these platforms are being…
AutoLog: A Log Sequence Synthesis Framework for Anomaly Detection
Yintong Huo, Yichen Li, Yuxin Su +3
The rapid progress of modern computing systems has led to a growing interest in informative run-time logs. Various log-based anomaly detection techniques have been proposed to ensu…
Hue: A User-Adaptive Parser for Hybrid Logs
Junjielong Xu, Qiuai Fu, Zhouruixing Zhu +4
Log parsing, which extracts log templates from semi-structured logs and produces structured logs, is the first and the most critical step in automated log analysis. While existing…
ROME: Testing Image Captioning Systems via Recursive Object Melting
Boxi Yu, Zhiqing Zhong, Jiaqi Li +3
Image captioning (IC) systems aim to generate a text description of the salient objects in an image. In recent years, IC systems have been increasingly integrated into our daily li…
Exploring the Effectiveness of LLMs in Automated Logging Generation: An Empirical Study
Yichen Li, Yintong Huo, Zhihan Jiang +5
Automated logging statement generation supports developers in documenting critical software runtime behavior. Given the great success in natural language generation and programming…
Validating Multimedia Content Moderation Software via Semantic Fusion
Wenxuan Wang, Jingyuan Huang, Chang Chen +5
The exponential growth of social media platforms, such as Facebook and TikTok, has revolutionized communication and content publication in human society. Users on these platforms c…
Incident-aware Duplicate Ticket Aggregation for Cloud Systems
Jinyang Liu, Shilin He, Zhuangbin Chen +10
In cloud systems, incidents are potential threats to customer satisfaction and business revenue. When customers are affected by incidents, they often request customer support servi…
Automated Testing of Image Captioning Systems
Boxi Yu, Zhiqing Zhong, Xinran Qin +3
Image captioning (IC) systems, which automatically generate a text description of the salient objects in an image (real or synthetic), have seen great progress over the past few ye…
AEON: A Method for Automatic Evaluation of NLP Test Cases
Jen-tse Huang, Jianping Zhang, Wenxuan Wang +3
Due to the labor-intensive nature of manual test oracle construction, various automated testing techniques have been proposed to enhance the reliability of Natural Language Process…
Empirical Standards for Software Engineering Research
Paul Ralph, Nauman bin Ali, Sebastian Baltes +39
Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e.g. a questionnaire survey). The ACM SIGSOFT Paper and Peer…
A Survey on Automated Log Analysis for Reliability Engineering
Shilin He, Pinjia He, Zhuangbin Chen +3
Logs are semi-structured text generated by logging statements in software source code. In recent decades, software logs have become imperative in the reliability assurance mechanis…
Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics
Jieming Zhu, Shilin He, Pinjia He +2
Logs have been widely adopted in software system development and maintenance because of the rich runtime information they record. In recent years, the increase of software size and…
Structure-Invariant Testing for Machine Translation
Pinjia He, Clara Meister, Zhendong Su
In recent years, machine translation software has increasingly been integrated into our daily lives. People routinely use machine translation for various applications, such as desc…
Tools and Benchmarks for Automated Log Parsing
Jieming Zhu, Shilin He, Jinyang Liu +4
Logs are imperative in the development and maintenance process of many software systems. They record detailed runtime information that allows developers and support engineers to mo…
A Directed Acyclic Graph Approach to Online Log Parsing
Pinjia He, Jieming Zhu, Pengcheng Xu +2
Logs are widely used in modern software system management because they are often the only data accessible that record system events at runtime. In recent years, because of the ever…
CARP: Context-Aware Reliability Prediction of Black-Box Web Services
Jieming Zhu, Pinjia He, Qi Xie +2
Reliability prediction is an important task in software reliability engineering, which has been widely studied in the last decades. However, modelling and predicting user-perceived…