PACER: Fine-grained Interactive Paper via Camera-touch Hybrid Gestures on a Cell Phone

Abstract

PACER is a gesture-based interactive paper system that
supports fine-grained paper document content manipulation
through the touch screen of a cameraphone. Using the
phone’s camera, PACER links a paper document to its
digital version based on visual features. It adopts camera-based
phone motion detection for embodied gestures (e.g.
marquees, underlines and lassos), with which users can
flexibly select and interact with document details (e.g.
individual words, symbols and pixels). The touch input is
incorporated to facilitate target selection at fine granularity,and to address some limitations of the embodied
interaction, such as hand jitter and low input sampling rate. This hybrid interaction is coupled with other techniques such as semi-real time document tracking and loose physical-digital document registration, offering a gesture-based
command system. We demonstrate the use of
PACER in various scenarios including work-related
reading, maps and music score playing. A preliminary user
study on the design has produced encouraging user
feedback, and suggested future research for better
understanding of embodied vs. touch interaction and one vs.
two handed interaction.