ML Day 2014 – “Perceptual Annotation”: from Biologically Inspired, to Biologically Informed Machine Learning

Many machine learning applications, explicitly or implicitly, attempt to mimic natural human abilities in a machine. Indeed, any setting where human-provided labels are used as ground truth – whether the system aspires to be biologically-inspired or not – is ultimately driven by the human visual and cognitive system and its ability to provide accurate exemplary labels. However, human-provided ground-truth labels are in many ways just the tip of the iceberg of the information that can be extracted from human judgments. I will describe a new approach – called “perceptual annotation” – in which we use an advanced online psychometric testing platform to acquire new kinds of human annotation data, and we incorporate these data directly into the formulation of a machine learning algorithm. A key intuition for this approach is that while it may be infeasible to dramatically increase the amount of data and high-quality labels available for the training of a given system, measuring the latent exemplar-by-exemplar landscape of difficulty and patterns of human errors can provide important information for regularizing the solution of the system at hand. Finally, I will conclude by exploring how this approach can be extended to incorporate an even greater diversity of different kinds of biological data