AutoSens: Inferring Latency Sensitivity of User Activity through Natural Experiments

We consider the problem of inferring the latency sensitivity of user activity in the context of interactive online services. Our method relies on natural experiments, i.e., leveraging the variation in user experienced latency seen in the normal course. At its core, our technique, dubbed AutoSens, compares the distribution of latency of the user actions actually performed with the underlying distribution of latency independent of whether users choose to perform any action. This then yields a normalized user preference based on latency. We discuss ways of mitigating various confounders and then present our findings in the context of a large online email service, Microsoft Outlook Web Access (OWA).