30 citations · 37 across the 14 of their papers we have counts for
4 papers · 1 filter
Running the Gauntlet: Hard Agentic Tasks
Mykola Vysotskyi, Runqi Lin, Grzegorz Biziel +21
As agentic systems continue to evolve and are widely deployed in real-world scenarios, there is a growing demand to faithfully evaluate their capabilities. However, current benchma…
Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
Olawale Salaudeen, Haoran Zhang, Kumail Alhamoud +2
Benchmarks for out-of-distribution (OOD) generalization frequently show a strong positive correlation between in-distribution (ID) and OOD accuracy across models, termed "accuracy-…
Real-Time Evaluation in Online Continual Learning: A New Hope
Yasir Ghunaim, Adel Bibi, Kumail Alhamoud +5
Current evaluations of Continual Learning (CL) methods typically assume that there is no constraint on training time and computation. This is an unrealistic assumption for any real…
Generalizability of Adversarial Robustness Under Distribution Shifts
Kumail Alhamoud, Hasan Abed Al Kader Hammoud, Motasem Alfarra +1
Recent progress in empirical and certified robustness promises to deliver reliable and deployable Deep Neural Networks (DNNs). Despite that success, most existing evaluations of DN…