7 papers
Automated Evaluation of Multi-Turn Dialogues in In-Car Conversational Assistants
Vaishnav Negi, Lev Sorokin, Soroosh Tayebi Arasteh +1
In-car conversational assistants (ICAs) are increasingly integrated into vehicles to support route planning, vehicle control, and information access. Ensuring their reliability is…
Diversity-Guided Search-Based Testing of Large Language Model Applications
Lev Sorokin, Ivan Vasilev, Ken E. Friedl +1
Large Language Model (LLM)-based applications are increasingly deployed across domains including customer service, education, and mobility. These systems are prone to inaccurate, f…
Search-based Testing of Vision Language Models for In-Car Scene Understanding
Lev Sorokin, Chen Yang, Ken E. Friedl +1
In the automotive domain, in-car scene understanding (ISU) enables the detection of safety-critical events, such as driver distraction, and supports drivers or passengers by analyz…
STELLAR: A Search-Based Testing Framework for Large Language Model Applications
Lev Sorokin, Ivan Vasilev, Ken E. Friedl +1
Large Language Model (LLM)-based applications are increasingly deployed across various domains, including customer service, education, and mobility. However, these systems are pron…
Benchmarking Contextual Understanding for In-Car Conversational Systems
Philipp Habicht, Lev Sorokin, Abdullah Saydemir +2
In-Car Conversational Question Answering (ConvQA) systems significantly enhance user experience by enabling seamless voice interactions. However, assessing their accuracy and relia…
Automated Factual Benchmarking for In-Car Conversational Systems using Large Language Models
Rafael Giebisch, Ken E. Friedl, Lev Sorokin +1
In-car conversational systems bring the promise to improve the in-vehicle user experience. Modern conversational systems are based on Large Language Models (LLMs), which makes them…
Large Language Models for Secure Code Assessment: A Multi-Language Empirical Study
Kohei Dozono, Tiago Espinha Gasiba, Andrea Stocco
Most vulnerability detection studies focus on datasets of vulnerabilities in C/C++ code, offering limited language diversity. Thus, the effectiveness of deep learning methods, incl…