4 papers
A Mixed-Behavior Vote Model for Multimedia Subjective Quality Votes, Means, and Variances
Jaden Pieper, Stephen D. Voran
The relationship between subjective test vote variance and vote mean (or MOS) is well-studied, and the mathematically admissible vote variance region has been previously defined. W…
Bounds on Agreement between Subjective and Objective Measurements
Jaden Pieper, Stephen D. Voran
Objective estimators of multimedia quality are often judged by comparing estimates with subjective "truth data," most often via Pearson correlation coefficient (PCC) or mean-square…
Unseen but not Unknown: Using Dataset Concealment to Robustly Evaluate Speech Quality Estimation Models
Jaden Pieper, Stephen D. Voran
We introduce Dataset Concealment (DSC), a rigorous new procedure for evaluating and interpreting objective speech quality estimation models. DSC quantifies and decomposes the perfo…
AlignNet: Learning dataset score alignment functions to enable better training of speech quality estimators
Jaden Pieper, Stephen D. Voran
We develop two complementary advances for training no-reference (NR) speech quality estimators with independent datasets. Multi-dataset finetuning (MDF) pretrains an NR estimator o…