Inking Outside the Box: How Context Sensing Affords More Natural Pen (and Touch) Computing

The authors were invited to present a reprise of a recently-published paper on Sensing Techniques for Tablet + Stylus Interaction at the WIPTTE 2014 Workshop. The talk took the original contribution as a point of departure, because for the WIPTTE venue we felt that the most important role of the work was to illuminate and help the audience understand more deeply the interaction modalities of pen and touch – as well as their use in tandem. And in the process the authors felt like they came to understood the topic more deeply as well, hence the paper that follows.

One of the premises of the talk was that even a concept as seemingly straightforward as ‘touch’ – not to mention pen + touch, used together in complementary roles – is perhaps not as well understood as we might think it is.

In particular, we argue that beyond the standard idiom of touch (and multi-touch) interaction on touchscreens, there are many aspects of ‘touch’ that are rarely considered (much less actually sensed) by existing devices and interaction designs. We show how this surrounding context of manual activity – how the user is holding the tablet, how the user is gripping the pen, and how each device is oriented and moving relative to the other – have the potential to considerably enrich interaction with tablets, and thereby to re-define what we conceive of as ‘natural’ interaction with pen and touch.