Grading video interviews with fairness considerations

Abstract

There has been considerable interest in predicting human emotions and traits using facial images and videos. Lately, such work has come under criticism for poor labeling practices, inconclusive prediction results and fairness considerations. We present a careful methodology to automatically derive social skills of candidates based on their video response to interview questions. We, for the first time, include video data from multiple countries encompassing multiple ethnicities. Also, the videos were rated by individuals from multiple racial backgrounds, following several best practices, to achieve a consensus and unbiased measure of social skills. We develop two machine-learning models to predict social skills. The first model employs expert-guidance to use plausibly causal features. The second uses deep learning and depends solely on the empirical correlations present in the data. We compare errors of both these models, study the specificity of the models and make recommendations. We further analyze fairness by studying the errors of models by race and gender. We verify the usefulness of our models by determining how well they predict interview outcomes for candidates. Overall, the study provides strong support for using artificial intelligence for video interview scoring, while taking care of fairness and ethical considerations.

arXiv commentSubmitted to NeurIPS2020

References in corpus (7)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Equality of Opportunity in Supervised Learning
• A Convex Framework for Fair Regression
• Deep Residual Networks with Exponential Linear Unit
• Fair Regression: Quantitative Definitions and Reduction-based Algorithms
• HireNet: a Hierarchical Attention Model for the Automatic Analysis of Asynchronous Video Job Interviews
• Detection and Mitigation of Bias in Ted Talk Ratings