Learning to Generalize for Complex Selection Tasks

Selection tasks are common in modern computer interfaces: we are often required to select a set of files, emails, data entries, and the like. File and data browsers have sorting and block selection facilities to make these tasks easier, but for complex selections there is little to aid the user without writing complex search queries. We propose an interactive machine learning solution to this problem called “smart selection,” in which the user selects and deselects items as inputs to a selection classifier which attempts at each step to correctly generalize to the user’s target state. Furthermore, we take advantage of our data on how users perform selection tasks over many sessions, and use it to train a label regressor that models their generalization behavior: we call this process learning to generalize. We then combine the user’s explicit labels as well the label regressor outputs in the selection classifier to predict the user’s desired selections. We show that the selection classifier alone takes dramatically fewer mouse clicks than the standard file browser, and when used in conjunction with the label regressor, the predictions of the classifier are significantly more accurate with respect to the target selection state.