Ablation-Corrected Evaluation of Attribution Maps in Echocardiographic Ejection-Fraction Models
arXiv:2607.13738
The paper audits attribution methods for deep video models that estimate left‑ventricular ejection fraction from echocardiography, showing that while the models correctly focus on the left ventricle spatially, they fail to attend to the clinically important end‑systolic and end‑diastolic frames.
Abstract
Attribution maps for echocardiographic ejection-fraction models are evaluated by their overlap with an expert left-ventricular annotation, compared against a chance level that is computed from an area ratio rather than measured. We measure it. Two architectures trained on the same task attain overlap at 3.55 and 4.20 times measured chance, an eighteen percent difference a reader would take as the size of the gap between them. It is not. Replacing the annotated ventricle with a composition-matched surrogate changes one model's prediction six times more than replacing an equal-area control region and the other's twice, a factor of three; on sixty-two percent of cases for the second, neither the ventricle nor the attributed region moves the prediction at all. The reliability of this measurement, from two independent intervention batches, is 0.90 to 0.98. Overlap does not track the difference: scored against an ablation-derived label of which cases the model depends on the ventricle for, it reaches an area under the curve of 0.52 and 0.67. The design of the ablation also determines its answer, since ablating only the two annotated frames leaves the ventricle indistinguishable from a control region while ablating across all frames does not. A pediatric cohort reproduces all of these. We give the measured chance level, the symmetric-ablation protocol, the reliability estimate, and the ablation-prediction score as things to report alongside overlap.
12 pages, 4 figures. v2: the chance level is now measured rather than computed, which corrects the ratios reported in v1. Adds a composition-matched ablation with an equal-area control, a reliability estimate for it, an ablation-derived score to report alongside overlap, seed repetition, and pediatric validation