No Interaction as Indicator of Search Satisfaction

At Bing, measuring user success has always been a deciding factor as to which feature or change is shipped to production. Testing such changes is carried out through randomized controlled experiments, where success metrics are used to measure the treatment effect on user satisfaction. Over the years, we have designed and refined our metrics to capture various user interactions, from search queries to clicks and hovers, and interpreted them to predict users’ satisfaction with the search engine. One of the main scenarios that is hard to interpret is search result page abandonment, where the user doesn’t click on the page or interact with any specific element. In this scenario of abandonment, we need to differentiate cases where the user abandoned due to getting the information they need without clicking on any results, from those where the user abandoned due to a defective and/or unsatisfactory search result page. In this talk, we outline Bing’s journey in addressing this measurement problem. We talk about our initial effort of considering the presence of specific elements on the page as indicator of success; to our offline/online hybrid approach to identify good abandonment; and finally, to a fully-online solution that relies on a user’s behavior across their search session. We also cover the pitfalls of the different approaches, how we evaluate them and the current challenges and problems left to solve.

View the slides from this presentation here: http://tinyurl.com/wsdm19-widad-machmouchi

日期：
演讲者：
Widad Machmouchi