4 papers
U-Space: Uncovering When and Why Uncertainty Arises in Language Models
Tobias Braun, Nils Loose, Alexander Herzog +4
Large language models are informing decisions with ever-higher stakes. As the consequences of their errors grow, a central question becomes harder to ignore: how much can we trust…
REPLICANT: Learning Policies for Evading and Hardening Malware Detectors
Shae McFadden, Ilias Tsingenopoulos, Mario D'Onghia +5
To determine the real-world effectiveness of machine learning based malware detection, it is vital to evaluate its robustness against highly capable adversaries. However, state-of-…
Reading Calibrated Uncertainty from Language Model Trajectories
Aliai Eusebi, Alexander Herzog, Xiaoyu Liang +3
The maximum softmax probability (MSP) represents a default approach when evaluating uncertainty quantification for language model generation with structured output. Although cheap,…
On the Reliability and Stability of Selective Methods in Malware Classification Tasks
Alexander Herzog, Aliai Eusebi, Lorenzo Cavallaro
The performance figures of modern drift-adaptive malware classifiers appear promising, but does this translate to genuine operational reliability? The standard evaluation paradigm…