Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input

Our research showcases a conceptual model for bimanual input that spans a wide range of form-factors including smartphones, slates, and tabletop systems using a combination of pen, touch, motion sensing, and voice input modalities. The pen writes, the hands manipulate, and the combination of modalities yields new tools – as well as compelling over-the-shoulder playback experiences for human-human communication.

Manual Deskterity is a prototype digital drafting table that supports both pen and touch input. We explore a division of labor between pen and touch that flows from natural human skill and differentiation of roles of the hands. We also explore the simultaneous use of pen and touch to support novel compound gestures.