4 papers
Challenges in Testing Large Language Model Based Software: A Faceted Taxonomy
Felix Dobslaw, Robert Feldt, Juyeon Yoon +1
Large Language Models (LLMs) and Multi-Agent LLMs (MALLMs) introduce non-determinism unlike traditional or machine learning software, requiring new approaches to verifying correctn…
Adaptive Testing for LLM-Based Applications: A Diversity-based Approach
Juyeon Yoon, Robert Feldt, Shin Yoo
The recent surge of building software systems powered by Large Language Models (LLMs) has led to the development of various testing frameworks, primarily focused on treating prompt…
Predictive Prompt Analysis
Jae Yong Lee, Sungmin Kang, Shin Yoo
Large Language Models (LLMs) are machine learning models that have seen widespread adoption due to their capability of handling previously difficult tasks. LLMs, due to their train…
Lachesis: Predicting LLM Inference Accuracy using Structural Properties of Reasoning Paths
Naryeong Kim, Sungmin Kang, Gabin An +1
Large Language Models are increasingly used to build agents to perform more complex tasks. As LLMs perform more complicated reasoning through longer interactions, self-consistency,…