Man versus Machine: Evaluating IVR versus a Live Operator for Phone Surveys in India

Many organizations in the developing world need to conduct phone surveys to collect data from low-income respondents. Such organizations generally have two options: employ a live operator, or utilize interactive voice response (IVR). Despite the relevance of this question, we are unaware of any work that rigorously compares the accuracy, speed, and cost of an IVR survey relative to a live operator.

In this paper, we address these questions by giving two identical interviews one using IVR, and one using a live operator to 31 low-income job seekers in India. The IVR interview included a brief introduction by a live operator, to provide context for the call. Out of the 20 people who completed both surveys, we found that IVR incurs a 4.0% error rate (95% C.I. 2.5% 6.1%) and requires 2.5 times longer for users. We summarize our experience as a set of recommendations for practitioners in this space.