1 citations · 1 across the 8 of their papers we have counts for
10 papers
Diversity-Guided Search-Based Testing of Large Language Model Applications
Lev Sorokin, Ivan Vasilev, Ken E. Friedl +1
Large Language Model (LLM)-based applications are increasingly deployed across domains including customer service, education, and mobility. These systems are prone to inaccurate, f…
Beyond Overt Reactions: Analyzing Subtle User Emotional Response to Unexpected In-Vehicle System Behavior
Huy Quyen Ngo, Suresh Kumaar Jayaraman, Brian Mok +4
Modern vehicles, with advanced AI voice and autonomous navigation features, extend beyond traditional driving but, like any autonomous system, can potentially make mistakes or beha…
Search-based Testing of Vision Language Models for In-Car Scene Understanding
Lev Sorokin, Chen Yang, Ken E. Friedl +1
In the automotive domain, in-car scene understanding (ISU) enables the detection of safety-critical events, such as driver distraction, and supports drivers or passengers by analyz…
LoCar: Localization-Aware Evaluation of In-Vehicle Assistants through Fine-Grained Sociolinguistic Control
Seogyeong Jeong, Kiwoong Park, Seyoung Song +4
While Large Language Models (LLMs) are increasingly integrated into in-vehicle conversational systems, identifying the optimal model remains challenging due to the lack of domain-s…
Automated structural testing of LLM-based agents: methods, framework, and case studies
Jens Kohl, Otto Kruse, Youssef Mostafa +9
LLM-based agents are rapidly being adopted across diverse domains. Since they interact with users without supervision, they must be tested extensively. Current testing approaches f…
STELLAR: A Search-Based Testing Framework for Large Language Model Applications
Lev Sorokin, Ivan Vasilev, Ken E. Friedl +1
Large Language Model (LLM)-based applications are increasingly deployed across various domains, including customer service, education, and mobility. However, these systems are pron…
Benchmarking Contextual Understanding for In-Car Conversational Systems
Philipp Habicht, Lev Sorokin, Abdullah Saydemir +2
In-Car Conversational Question Answering (ConvQA) systems significantly enhance user experience by enabling seamless voice interactions. However, assessing their accuracy and relia…
Automated Factual Benchmarking for In-Car Conversational Systems using Large Language Models
Rafael Giebisch, Ken E. Friedl, Lev Sorokin +1
In-car conversational systems bring the promise to improve the in-vehicle user experience. Modern conversational systems are based on Large Language Models (LLMs), which makes them…
InCA: Rethinking In-Car Conversational System Assessment Leveraging Large Language Models
Ken E. Friedl, Abbas Goher Khan, Soumya Ranjan Sahoo +3
The assessment of advanced generative large language models (LLMs) poses a significant challenge, given their heightened complexity in recent developments. Furthermore, evaluating…
CarExpert: Leveraging Large Language Models for In-Car Conversational Question Answering
Md Rashad Al Hasan Rony, Christian Suess, Sinchana Ramakanth Bhat +6
Large language models (LLMs) have demonstrated remarkable performance by following natural language instructions without fine-tuning them on domain-specific tasks and data. However…