CARETS: A Consistency And Robustness Evaluative Test Suite for VQA

Carlos E. Jimenez, Olga Russakovsky, Karthik Narasimhan

We introduce CARETS, a systematic test suite to measure consistency and robustness of modern VQA models through a series of six fine-grained capability tests. In contrast to existi…