When Inputs Receive No Score: Evaluating Open-Web Event Admission on the Complete Stream

Abstract

A Web item can link to the correct event without contributing information worth organizing. Evaluating item-level admission therefore begins with complete inputs, including those a processing pipeline fails to score. We study how this upstream selection changes ranking comparisons and constrains operational decisions. Across six instruction-model backbones, three strategies, and 602 human-annotated evaluation items, systems score different populations with different class compositions. Comparing rankings on shared inputs separates these population differences from discrimination. Score availability also limits attainable retention: four configurations score only ten of twelve positive development items, making a 90% full-stream target unreachable by threshold adjustment. Paired precision experiments show that this constraint depends on execution configuration as well as model choice. Qwen scores the same number of evaluation items under INT4 and INT8, yet exchanges four scored items for four others. Extraction-recovery diagnostics further show that obtaining more scores can increase false acceptance under an unchanged threshold. We connect these observations through a complete-stream evaluation protocol that checks calibration feasibility and separates upstream score loss from decision loss. The resulting account explains which inputs a system can score, what its operating point can achieve, and what it actually admits.

Revised manuscript; 4 figures, 4 tables; expanded six-backbone evaluation and paired inference-precision experiments