Tracking Users’ Clicks and Submits: Tradeoffs between User Experience and Data Loss

Tracking users’ online clicks and form submits (e.g., searches) is critical for web analytics, controlled experiments, and
business intelligence. Most sites use web beacons to track user actions, but waiting for the beacon to return on clicks
and submits slows the next action (e.g., showing search results or the destination page). One possibility is to use a short
timeout and common wisdom is that the more time given to the tracking mechanism (suspending the user action), the
lower the data loss. Research from Amazon, Google, and Microsoft showed that small delays of a few hundreds of
milliseconds have dramatic negative impact on revenue and user experience (Kohavi, et al., 2009 p. 173), yet we found
that many websites allow long delays in order to collect click. For example, until March 2010, multiple Microsoft sites
waited for click beacons to return with a 2-second timeout, introducing a delay of about 400msec on user clicks. To the
best of our knowledge, this is the first published empirical study of the subject under a controlled environment. While
we confirm the common wisdom about the tradeoff in general, a surprising result is that the tradeoff does not exist for
the most common browser family, Microsoft Internet Explorer (IE), where no delay suffices. This finding has significant
implications for tracking users since no waits is required to prevent data loss for IE browsers and it could significantly
improve revenue and user experience. The recommendations here have been implemented by the MSN US home page
and Hotmail.