A Human in the Loop is Not Enough: The Need for Human-Subject Experiments in Facial Recognition

The deployment of facial recognition systems in high-stakes scenarios has sparked widespread concerns about privacy, fairness, and accountability. A common response to these concerns is the suggestion of adding a human in the loop to provide oversight and ensure fairness and accountability. However, the effectiveness of this approach is seldom studied empirically, and humans are known to have biases of their own. In this position paper, we argue for the necessity of empirical studies of human-in-the-loop facial recognition systems. We outline several technical and ethical challenges that arise when conducting such empirical studies and when interpreting their results. Our goal is to initiate a discussion about ways for AI and HCI researchers to work together on human-centered approaches to empirically studying human-in-the-loop facial recognition systems.