Overcoming Failures of Imagination in AI Infused System Development and Deployment

NeurIPS 2020 requested that research paper submissions include impact statements on ‘potential nefarious uses and the consequences of failure.’ When researching, designing, and implementing systems, a key challenge to anticipating risks, however, is to overcome what Clarke (1962) called ‘failures of imagination.’ The growing research on bias, fairness, and transparency in computational systems aims to illuminate and mitigate harms, and could thus help inform reflections on possible negative impacts of particular pieces of technical work. The prevalent notion of computational harms — narrowly construed as either allocational or representational harms — does not fully capture the context dependent and unobservable nature of harms across the wide range of AI infused systems. The current literature primarily addresses only a small range of examples of harms to motivate algorithmic fixes, overlooking the wider scope of probable harms and the way these harms may affect different stakeholders. The system affordances and possible usage scenarios may also exacerbate harms in unpredictable ways, as they determine stakeholders’ control (including non-users) over how they interact with a system output. To effectively assist in anticipating and identifying harmful uses, we argue that frameworks of harms must be context-aware and consider a wider range of potential stakeholders, system affordances, uses, and outputs, as well as viable proxies for assessing harms in the widest sense.