97 citations · 440 across the 25 of their papers we have counts for
29 papers
Predicting Safety Misbehaviours in Autonomous Driving Systems using Uncertainty Quantification
Ruben Grewal, Paolo Tonella, Andrea Stocco
The automated real-time recognition of unexpected situations plays a crucial role in the safety of autonomous vehicles, especially in unsupported and unpredictable scenarios. This…
Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study
Luca Giamattei, Matteo Biagiola, Roberto Pietrantuono +2
In a recent study, Reinforcement Learning (RL) used in combination with many-objective search, has been shown to outperform alternative techniques (random search and many-objective…
Boundary State Generation for Testing and Improvement of Autonomous Driving Systems
Matteo Biagiola, Paolo Tonella
Recent advances in Deep Neural Networks (DNNs) and sensor technologies are enabling autonomous driving systems (ADSs) with an ever-increasing level of autonomy. However, assessing…
Neural Embeddings for Web Testing
Kasun Kanaththage, Luigi Libero Lucio Starace, Matteo Biagiola +2
Web test automation techniques often rely on crawlers to infer models of web applications for automated test generation. However, current crawlers rely on state equivalence algorit…
Testing of Deep Reinforcement Learning Agents with Surrogate Models
Matteo Biagiola, Paolo Tonella
Deep Reinforcement Learning (DRL) has received a lot of attention from the research community in recent years. As the technology moves away from game playing to practical contexts,…
Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testing
Matteo Biagiola, Andrea Stocco, Vincenzo Riccio +1
Simulation-based testing represents an important step to ensure the reliability of autonomous driving software. In practice, when companies rely on third-party general-purpose simu…
Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks
Michael Weiss, Paolo Tonella
Recent decades have seen the rise of large-scale Deep Neural Networks (DNNs) to achieve human-competitive performance in a variety of artificial intelligence tasks. Often consistin…
Repairing DNN Architecture: Are We There Yet?
Jinhan Kim, Nargiz Humbatova, Gunel Jahangirova +2
As Deep Neural Networks (DNNs) are rapidly being adopted within large software systems, software developers are increasingly required to design, train, and deploy such models into…
When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study
Vincenzo Riccio, Paolo Tonella
Testing Deep Learning (DL) based systems inherently requires large and representative test sets to evaluate whether DL systems generalise beyond their training datasets. Diverse Te…
Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelines
Michael Weiss, Paolo Tonella
Deep Neural Networks (DNN) are increasingly used as components of larger software systems that need to process complex data, such as images, written texts, audio/video signals. DNN…
Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testing
Michael Weiss, André García Gómez, Paolo Tonella
Deep Neural Networks (DNNs) are becoming a crucial component of modern software systems, but they are prone to fail under conditions that are different from the ones observed durin…
Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)
Michael Weiss, Paolo Tonella
Test Input Prioritizers (TIP) for Deep Neural Networks (DNN) are an important technique to handle the typically very large test datasets efficiently, saving computation and labelin…
Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems
Andrea Stocco, Brian Pulfer, Paolo Tonella
Safe deployment of self-driving cars (SDC) necessitates thorough simulated and in-field testing. Most testing techniques consider virtualized SDCs within a simulation environment,…
DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score
Vincenzo Riccio, Nargiz Humbatova, Gunel Jahangirova +1
Deep Learning (DL) components are routinely integrated into software systems that need to perform complex tasks such as image or natural language processing. The adequacy of the te…
DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search
Tahereh Zohdinasab, Vincenzo Riccio, Alessio Gambi +1
Deep Learning (DL) has been successfully applied to a wide range of application domains, including safety-critical ones. Several DL testing approaches have been recently proposed i…
A Review and Refinement of Surprise Adequacy
Michael Weiss, Rwiddhi Chakraborty, Paolo Tonella
Surprise Adequacy (SA) is one of the emerging and most promising adequacy criteria for Deep Learning (DL) testing. As an adequacy criterion, it has been used to assess the strength…
GAssert: A Fully Automated Tool to Improve Assertion Oracles
Valerio Terragni, Gunel Jahangirova, Paolo Tonella +1
This demo presents the implementation and usage details of GASSERT, the first tool to automatically improve assertion oracles. Assertion oracles are executable boolean expressions…
Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring
Michael Weiss, Paolo Tonella
Modern software systems rely on Deep Neural Networks (DNN) when processing complex, unstructured inputs, such as images, videos, natural language texts or audio signals. Provided t…
Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification
Michael Weiss, Paolo Tonella
Uncertainty and confidence have been shown to be useful metrics in a wide variety of techniques proposed for deep learning testing, including test data selection and system supervi…
Deep Reinforcement Learning for Black-Box Testing of Android Apps
Andrea Romdhana, Alessio Merlo, Mariano Ceccato +1
The state space of Android apps is huge and its thorough exploration during testing remains a major challenge. In fact, the best exploration strategy is highly dependent on the fea…
An Empirical Study on Failed Error Propagation in Java Programs with Real Faults
Gunel Jahangirova, David Clark, Mark Harman +1
During testing, developers can place oracles externally or internally with respect to a method. Given a faulty execution state, i.e., one that differs from the expected one, an ora…
Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing
Vincenzo Riccio, Paolo Tonella
With the increasing adoption of Deep Learning (DL) for critical tasks, such as autonomous driving, the evaluation of the quality of systems that rely on DL has become crucial. Once…
A Framework for In-Vivo Testing of Mobile Applications
Mariano Ceccato, Davide Corradini, Luca Gazzola +4
The ecosystem in which mobile applications run is highly heterogeneous and configurable. All layers upon which mobile apps are built offer wide possibilities of variations, from th…
Taxonomy of Real Faults in Deep Learning Systems
Nargiz Humbatova, Gunel Jahangirova, Gabriele Bavota +3
The growing application of deep neural networks in safety-critical domains makes the analysis of faults that occur in such systems of enormous importance. In this paper we introduc…