Description

This study explores three hand-interaction techniques, including the gaze and pinch, touch and grab, and worlds-in-miniature interaction. Overall, a comparative analysis reveals that the WIM provided the best usability and task performance than other studied techniques. We also conducted in-depth interviews and analyzed participants’ hand gestures. Gesture analysis reveals that shapes of furniture, as well as its perceived features such as weight, largely determined the participant’s instinctive form of hand interaction. Based on these findings, we present design suggestions that can aid 3D interaction designers to develop a natural hand interaction for mixed reality.

The use of hand input has received increasing attention in the virtual reality (VR) community. There is a growing body of research that has begun to cast light on hand-interaction; yet, many prior VR research predominantly focused on 3D interaction using controllers. Using bare-hands offers a distinctive user experience as opposed to using controllers. Compared to the controller where a single button is often mapped to a specific action, human hands are capable of various gestures by nature. For example, to move a virtual object, the standard design practice for VR controllers is to hold a grip button. The dynamic nature of human hands, however, makes it challenging to find and pin down the most natural hand input among possible gestures—lift, grab, push, point, pinch, and more. The fundamental difference lies in our familiarity. Unlike a controller that typically requires extra efforts of learning, we are more likely to adopt our habitual actions of using hands when interacting with the virtual object. Consequently, identifying the most natural hand interaction is a complicated process since the idea of “natural” hand interaction can vary depends on the context, and it requires an understanding of how we interact with objects in everyday life.

Prior 3DUI research has developed the novel metaphors for hand gestures such as finger-pointing that works as a mouse cursor [10,15, 27, 39], a multimodal interaction that integrates finger-pointing and gaze [21,32,42], and multiple gestures such as grabbing and pinching for complex tasks [16, 20]. While these streams of research provide valuable insights in designing innovative hand interactions, relatively little is known on tradeoff among existing technologies. Furthermore, the majority of prior works are focused on the usability aspects of hand input. For instance, 3DUI research abounds with discussions on efficient and accurate hand inputs that address technical concerns such as object occlusions and small field of view [13,17,21,32]. The question of which hand gesture would render a natural interaction, on the other hand, remains less explored.

To this end, this study aims to provide a comprehensive review of the tradeoff among the well-known 3D hand interaction designs. We compare three 3D interactions; 1) multimodal interaction using gaze and pinch gesture, 2) direct touch and grab interaction, and 3) worldsin-miniature. In comparing these three interactions, we first observed participants’ behavioral responses without giving instructions. This approach helps us to examine the discoverability of each interaction design, thereby providing insights into natural hand interaction. We particularly focused on an interaction that involves selecting and arranging furniture items. Architecture and interior design are one of the most promising application areas in MR; thus findings from this study can provide immediate real-world implications.