Report: State of the Art Seminar

Explorations on Body-Gesture based Object Selection on HMD based

VR Interfaces for Dense and Occluded Dense Virtual Environments

By

Shimmila Bhowmick (Roll No. 166105005)

Guided by Dr. Keyur Sorathia

Department of Design Indian Institute of Technology, Guwahati,

Assam, India November 2018

Table of contents

1. Abstract 1

2. Introduction to Virtual Reality (VR) 2

2.1 Definition of VR 2

2.2 Advantages of VR interfaces 2

2.3 Exploration of VR application in recent years 3

3. Introduction to Object Selection 4

3.1 What is object selection? 4

3.2 Importance of object selection in VR interfaces 5

4. Literature review on input methods for object selection 10

4.1 Controller-based input techniques for object selection 10

4.1.1 Research gap in controller based object selection method 10

4.2 Controller-less input techniques for object selection 11

4.2.1 Research gap in controller-less object selection method 15

5. Research Questions 16

6. Methodology 17

7. References 18

Figures Figure 1. (a) Ray casting selection, the first object intersected by the ray is selected. (b) The depth ray selects the

intersected target which is closest to the depth marker by moving the hand forward and backward.

Figure 2. (a) the flexible pointer selecting a partially occluded object. (b) Cone Selection in which multiple objects fall in the selection cone however the object that is closest to the line (Object D) is the one selected. (c) Shadow Cone Selection is effected by choosing the object that is within all the cones projected from the hand (Object G). In this case, both Objects E and G are highlighted at the start, but E is dropped as the hand moves to its end position and direction.

Figure 3. (a) the conic volume of the aperture selection described from eye point and the aperture circle from the location of the eye. (b) the activation area of Dynaspot coupled to cursor speed. Multiple objects intersected and the object near the cursor is selected and highlighted.

Figure 4 (a) Region examination with initial and extended cone casting. (b) Sticky-ray example during translation of an input device from left to right.

Figure 5 (a) Experimental setup with Sphere-casting refined by QUAD-menu (SQUAD). It uses a progressive refinement technique for the highly cluttered environment. (b) Flower ray in which all the intersected targets are highlighted, flowered out in a marking menu and the desired target is selected.

Figure 6 (a) The Bubble cursor dynamically resizing its activation area depending on the proximity of surrounding

targets to select one target. (b) Double, bound and depth bubble cursor. Objects are selected in accordance with

a depth sphere, 2D ring, and cursor at center.

Figure 7 (a) A third person’s view of the Head-Crusher technique in which the object is between the user's finger and thumb in its image plane. (b) Sticky Finger technique uses the outstretched finger gesture to select objects in the image plane. (c) Zoom technique zooms in the area inside the cursor hence providing the user with a larger area to aim at. (d) Expand Technique uses zoom and SQUAD to select objects.

Figure 8. Scope selection technique with cursor activation area small and large with slow user input (left) and fast user input (right) simultaneously.

Figure 9. The user selects a highest ranking object which is an occluded, moving and far-away object using the bending ray.

Figure 10. Shapes detected when aligning multi-finger raycast projections.

Figure 11 (a) Hand extension metaphor employs a hand avatar in 3D for grabbing objects.

Figure 12 (a) Selection of occluded objects using Isith technique (b) AirTap is a “down and up” gesture of the index finger. Thumb Trigger is an “in and out” gesture of the thumb.

Figure 13 (a) selecting an object using magnetic force moves the object to hand. (b) selecting an object using innate pinch using pinch gesture

Figure 14. (a) Menu cone selects the intended target (in yellow) along with other targets. (b) A pull gesture confirms the selection and a menu appears. (c) A directional gesture is performed to select the target.

Figure 15 (a) Selection cone intersecting various objects. (b) refinement phase, moving the user to objects (c) single object selection. (d) returning to an original position with the object selected.

Figure 16. IDS method adopts a proximity sphere along the path described by the motion of the hand, and the objects that are intersected by it are considered for selection.

Figure 17. Movement of EZCursor VR using Head-movement and/or external input device.

1

1. Abstract

Recent years have seen incredible growth of Virtual Reality (VR) interfaces due to its capability

to simulate the real-world, provide an experience of the unseen, communicate a difficult concept with

ease and increase self-efficacy and learnability in the context of training and education. Moreover, VR

interfaces presented on Head Mounted Displays (HMDs) present the advantages of improved learning

outcomes, confidence, positive behavior and scaling to resource-constrained users and communities.

Despite these advantages, VR interfaces, especially HMD based VR interfaces possess challenges of

poor usability and usable user interface methods for object selection in a virtual environment (VE).

Although numerous studies have investigated controller based object selection for varied VEs, they

still demand holding a physical device that is cumbersome, increases fatigue and restricts user’s

movement in a physical world. Moreover, the inconsistent technology platforms and related input

devices demand users to learn the interaction each time they adopt a different technology platform.

To overcome the challenges of controller-based input interactions, researchers have also explored

controller-less input methods, especially object selection using body-gesture. However, the current

literature is limited in investigating the effectiveness of body-gestures for object selection in different

VEs including dense and occluded dense VE, varied object sizes, proximity, and distances. Moreover,

the current work employs a developer drive gesture design approach instead of a user-centric

approach, which has seen increased acceptance and adoption in HCI literature. The use of context and

user-centric body gestures for object selection of varied object density, object proximity, object size

and object distances in HMD based VR interfaces still remains unexplored and needs suitable

interventions.

Within this view, this research document presents 2 core research questions (RQ) and 3 sub-

questions for each core RQ with an aim to pursue them during the course of this Ph.D. It also presents

an overview of the methodology to investigate the proposed 2 core RQ and 2 sub-questions. This state

of the art seminar document starts with introducing VR, benefits of VR and its application areas in

recent years. This is followed by a definition of object selection, the importance of object selection

and factors that influence object selection in a VR interface. Further, a detailed literature review of

the controller based and controller-less object selection is presented along with their limitations and

research gaps. It includes the review of desktop based and projection based VR interfaces followed by

an investigation on HMD based VR interfaces. At last, the document details of out the 2 RQ and 3 sub-

questions followed by an overview of the methodology to conduct future studies.

2

2. Introduction to Virtual Reality

2.1. Definition of VR

Merriam-Webster dictionary 2018 (Merriam-Webster, 2018), defines Virtual Reality (VR) as

an artificial environment which is experienced through sensory stimuli (such as sights and sounds)

provided by a computer and in which one's actions partially determine what happens in the

environment. VR is also defined as a computer-generated digital environment that can be

experienced and interacted with as if that environments were real (Jerald, 2015).

2.2. Advantages of VR interfaces

A defining feature of VR is the ability to select and manipulate virtual objects interactively,

rather than simply viewing a passive environment (Bowman and Hodges 1997). It also enables

navigation, including travel and wayfinding into a virtual environment (VE) through various interactive

methods. This increases spatial experience, immersion, and presence, hence enhancing the overall

user experience. The advantages of VR over conventional digital interfaces are that the user is given

an opportunity to experience subject matter in a way that would be difficult if not impossible to

illustrate or describe through conventional methods. It has shown potential through improved

knowledge gain and self-efficacy in the context of training and education (Buttussi and Chittaro 2018;

Hui 2017; Li et al., 2017). It simulates a real-world environment, demonstrates difficult concepts with

ease (Lockwood 2004; Mikropoulos 2006), and provides control to the viewing contexts (Minocha et

al., 2017). Further, VEs accessed through Head Mounted Displays (HMDs) have shown improved

learning outcomes (Gutierrez 2007; Mikropoulos 2006), increased confidence (Yee and Bailenson

2007; Yee et al., 2009) and positive behavior change (Svoronos 2010). The increasing availability of

the low-cost viewers (such as Google Cardboard viewer) is creating opportunities to adapt to VR’s

potential in training and education, including solutions for underserved communities (Robertson,

2017).

2.3. Exploration of VR application in recent years

Historically, technologies supporting VR interfaces have evolved over the last two decades. It

evolved from desktop and multi-projection VR interfaces to sophisticated HMD based VR interfaces in

recent years. Although HMD based VR interfaces have been explored in early 1990, they faced

challenges of poor latency which results in nausea, fatigue and overall poor user experience. Advanced

HMDs such as HTC Vive, Oculus Rift etc. have largely solved these challenges. Additionally, their ability

to leverage up to 6 degrees of freedom (DoF), accurate tracking system and increased field of view

(FoV) provides a wide variety of advantages over traditional desktop and projection based interfaces.

The commercial success of 6 DoF HMDs and mobile phone-based low-cost HMDs (e.g. google

cardboard etc.) supported through a plethora of VR applications have dramatically increased the

popularity of VR interfaces in recent years. They have been extensively used in applications including

architectural and interior design, industrial training and safety procedures, fear removing applications,

prototyping, psychiatric treatment, scientific visualization, cultural heritage, virtual tourism and

collaborative work spaces across the world. Despite the above mentioned advantages and

applications in a variety of domains, the lack of usable user interfaces for VEs has been a major factor

preventing the large acceptance and adoption of VR systems. Most common problem observed in VR

interfaces is the absence of context and user centric object selection method, which is a primary and

the most commonly used feature for any VR interface. This problem is further amplified for a dense

VE and occluded dense VE, virtual objects with varied distances, sizes and proximity which is still an

inadequately explored challenge. Moreover, issues of the non-standardized technology platform and

3

input devices (especially for HMD based VR interfaces), unnatural input interaction methods and

inaccessibility of suitable object selection guidelines reduce the usability and hence, overall user

experiences. There is a strong need to study natural input interaction methods for objection selection

in VR interfaces that are user-centric, adequately supports challenges of different VEs and are

independent of various HMD based technology platforms in order to apply these input methods to a

wide variety of devices.

4

3. Introduction to Object Selection

3.1. What is an object selection?

The selection has been repeatedly identified as one of the fundamental task (Mine, 1995).

and the primary task for most common user’s interactions in a VE (Bowman et al., 2004). Selection is the process of indicating a target object for subsequent action including position, orientation, and information related to the object's properties. This ability to choose which object is the target for subsequent actions precedes any further behavior.

3.2. Importance of object selection in VR interfaces

Bowman et al., 2004, proposed a task-driven taxonomy to classify 3D interaction techniques according to four main interaction tasks in any VE: selection, manipulation, navigation, and application control. While manipulation, navigation and application control involves transformations to 3D objects (translations and rotations), modification of the current viewpoint and sending specific commands respectively, they can only be performed post object selection. Hence, selection is the primary task among all of them. Moreover, they suggest that improvements in selection tasks will also improve manipulation, application control, and navigation, as they are often preceded by selection tasks. In this sense, efficient and error-proof selection techniques are critical because they allow the user to control the interaction flow between the above tasks.

There are different categories of object selection for a VE. Argelaguet and Andujar, 2013, classified selection techniques according to the following 9 criteria: selection tool, tool control, selection tool DoFs, control­display ratio, motor and visual space relationship, disambiguation mechanism, selection trigger, and feedback methods. In addition to these, factors that affect object selection are different VEs including sparse, dense and occluded dense VE; object size; target proximity – how close and far are objects to each other; target density – how many objects are placed in a scene; multiple object selection – whether multiple objects can be selected at one; object distance- how far is the object in the scene; static and dynamic object – whether object is a moving object or a static object; and the speed of the object. Selecting dynamic objects is always more challenging than selecting static targets. The difficulty of the selection increases while (1) target size decreases, (2) target velocity increases and (3) target density increases. The selection is even more difficult in 3D environments because a target of interest can be occluded by others targets.

5

4. Literature review on input techniques for object selection

Selection involves indicating and performing the task at hand. While indication can be given

by the system, the importance lies in the selection technique to execute the task. This section presents

the literature review of the two most commonly explored selection techniques (a) controller based

input technique and (b) controller-less input technique for object selection in a 3D user interfaces and

VR interfaces. This section includes input technique applied in all technology platforms, including

desktop based VR, single and multi-projection based VR interfaces and HMD based VR interfaces. The

objective to include all technology platforms is to cover all potential object selection techniques

applied for all 3D user interfaces in the past two decades.

4.1. Controller-based input techniques for object selection

Ray-casting (Mine, 1995) is the most popular and earliest implementations of selection for virtual environments (VEs). Ray casting is shown in figure 1(a). Ray-casting requires only two degrees of freedom and works at any distance. Even though ray-casting provides better performance than virtual hand techniques in many situations, it also has limitations. When the visual size of the target is small, due to object size, occlusion, or distance from the user, ray-casting is slow and error-prone (Steed and Parker, 2004). Also, accurate selection with ray-casting in these situations requires a great deal of effort on the part of the user and may require the user to move closer to the object to be able to select it all. Other forms of raycasting metaphors have also been developed, such as Liang and Green’s 1994, ‘‘laser gun’’ ray-casting technique. With this technique, a ray is emitted from the user’s hand, so the user has control over the origin and trajectory of the ray, much like using a physical laser pointer. One observed problem with this technique was that it was difficult to select occluded, distant and small objects.

The depth ray and lock ray (Grossman and Balakrishnan, 2006), resolves the occlusion problem by augmenting the ray cursor with a depth marker, visualized as a small sphere, existing along the length of the ray. The user controls a depth marker by moving the hand forward or backward. The object intersected by the ray cursor, which is closest to the depth marker, can be selected. Depth ray technique is shown in figure 1(b). The lock ray is similar to depth ray but employs a button to clicks and hold to avoid ambiguity.

Figure 1(a) Ray casting selection, the first object intersected by the ray is selected. (b) The depth ray selects the

intersected target which is closest to the depth marker by moving the hand forward and backward.

Even though these techniques improve selection performance in general, they can have a

negative effect in cluttered environments and for occluded targets. Most of these techniques also obstruct the user’s view of the scene with the pointer or the user’s hand. The flexible pointer proposed by Olwal and Feiner 2003 allows the user to point around objects with a curved arrow, to select fully or partially obscured objects, and to point out objects of interest more clearly to other users in a collaborative environment. It adopts bendable ray as a selection tool to select objects. The flexible pointer is shown in figure 2(a). Cone-casting (Liang and Green, 1993), for example, extends ray-casting by replacing the ray with a cone-shaped volume to make it easier to select small or distant objects. In cluttered environments, however, many objects will fall inside the cone, so that the user still has to point precisely to select the desired object. Shadow cone-casting (Steed and Parker, 2004)

6

is a technique to select multiple objects. They can both be used in a mode where all objects that are intersected when activated or are closest to the ray are selected. All the objects are selected, but as the user moves their hand only objects that are always within the cone are selected when the button is released. While useful for group selections, the shadow cone does not provide a disambiguation mechanism, as all targets which are intersected during the entire selection period will be selected. This reduces accuracy and increases errors during object selection. Cone casting and shadow cone casting are shown in figure 2 (b) and (c) respectively.

Figure 2(a) the flexible pointer selecting a partially occluded object. (b) Cone Selection in which multiple objects fall

in the selection cone however the object that is closest to the line (Object D) is the one selected. (c) Shadow Cone

Selection is effected by choosing the object that is within all the cones projected from the hand (Object G). In this

case, both Objects E and G are highlighted at the start, but E is dropped as the hand moves to its end position and

direction.

Another technique called ‘‘spotlight selection’’ (Liang and Green, 1994) uses a similar

phenomenon in which the user emits a conic selection area, originating from the user’s hand. However, spotlight selection technique requires some method for disambiguating which object the user wants to select when multiple objects fall within the selected volume. These techniques, however, do not cater to a specific selection of occluded objects. There have also been a number of iterations on the ray casting metaphor, such as aperture based selection and aperture with orientation method (Forsberg et al., 1996). As shown in figure 3 (a), they are a modification of the spotlight selection technique in which the apex of the conic selection volume is set to the location of the participant’s dominant eye and the direction vector of the cone is the vector from that eye. The size of the selection volume is determined by the distance between the eye point and the aperture cursor. A participant can adjust the scope of the selection by moving her hand in and out, thus changing this distance. If multiple objects fall within the conic volume of the aperture, objects are selected whose orientation most closely matches the orientation of the tracker. Orientation information provides the primary disambiguation metric. If all candidate objects have similar orientations, the basic aperture technique disambiguation metric resorts. Dynaspot (Chapuis et al., 2009), a new type of area cursor, couples the cursor’s activation area with its speed. The speed-dependent behavior of DynaSpot allows a maximum spot size during the highest speed and also to decrease to just 1, thus becoming Raycast. When multiple objects are selected, the object nearest to the cursor is highlighted and selected.

Figure 3 (a) the conic volume of the aperture selection described from eye point and the aperture circle from the

location of the eye. (b) the activation area of Dynaspot coupled to cursor speed. Multiple objects intersected and the

object near the cursor is selected and highlighted.

7

PORT (Lucas, 2005), allows the selection of multiple objects and uses a series of movement

and resizing actions to define the set of targets. Benko and Feiner 2007, employed the ballon selection technique, which allows users to control the depth of selection over a 2D touch surface by using the distance between two hands. Steinicke et al., 2006, introduced region examination and sticky ray interaction metaphor for object selection in VEs (figure 4 (a) and (b)). Objects within the cone-casted region are considered and depending on the distance from the center of ray the nearest object is selected. If no object falls within the region an enlargement process of the target is performed and repeated until an intersection is found. In the sticky ray techniques, the first object to be hit by the ray becomes the active object and remains active until the virtual ray hits another selectable object.

Figure 4 (a) Region examination with initial and extended cone casting. (b) Sticky-ray example during translation of

an input device from left to right.

However, in highly cluttered environments these techniques require users to interact very

carefully to accomplish selection, and may actually result in worse performance than standard ray-casting in some situations. These techniques are desired only for singular hit and do not stipulate selection in the occluded environment. To address these challenges, selection methods that use a progressive refinement of the set of selectable objects have been proposed.

Progressive refinement requires a process of selection, using multiple steps. It uses imprecise actions to accomplish a task and the selection can be accurate, effortless but time taking. SQUAD (Kopper et al., 2006), uses a modified version of ray-casting that casts a sphere onto the surface to determine the selectable objects. The indicated objects are split into four groups through a QUAD-menu user interface. The user performs repeated selections until the selection contains a single object. SQUAD technique uses several discrete steps to iteratively select an object within a group of interest and was designed for selection of occluded objects in cluttered environments. SQUAD technique is shown in figure 5 (a). The flower ray technique (Grossman and Balakrishnan, 2006), in which multiple targets are concurrently selected by ray-casting and disambiguated in a second phase by a marking menu. Although this technique is suited for highly cluttered environments, it requires high precision for the ray selection and does not scale well to a large number of objects. This technique is shown in figure 5 (b).

Figure 5 (a) Experimental setup with Sphere-casting refined by QUAD-menu (SQUAD). It uses a progressive

refinement technique for the highly cluttered environment. (b) Flower ray in which all the intersected targets are

highlighted, flowered out in a marking menu and the desired target is selected.

An area cursor is a cursor that has a large activation area and has been found to perform better than other regular cursors for some target acquisition tasks. Worden et al. 1997, propose an enhanced area cursor by including a single point hotspot centered within the area cursor, which takes effect when more than one target is within the cursor’s boundary. The enhanced area cursor performed

8

identically to regular point cursors when targets were close together, and outperformed point cursors when targets were far apart. Enhanced area cursors can be used to disambiguate selection (Findlater et al., 2010). This enhanced area cursor is the major inspiration for the bubble cursor. The Bubble-cursor (Grossman and Balakrishnan, 2005) improves upon area cursors by dynamically resizing its activation area depending on the proximity of surrounding targets, such that only one target is selectable at any time. In the 2D technique, it dynamically resizes a circular cursor so that it only contains one object. The bubble cursor technique is shown in figure 6 (a). A 3D extension of the bubble-cursor, which uses a sphere instead of a circle, was presented by Vanacken et al. 2007. These techniques may actually perform worse in cluttered environments since even small movements will cause the ray or the cursor to constantly resize to select new targets. Cockburn and Firth, 2004, developed a similar technique based on expanding targets called bubble targets. Instead of increasing the size of the entire target, a bubble would appear around the target as the cursor approached. Argelaguet and Andujar, 2008, proposed a technique by dynamically scaling potential targets and by using depth-sorting to disocclude potential targets for selection. Rosa and Nagel 2010, defined bubble cursor variations double, bound and depth (figure 6(b)). The double bound cursor has two semi-transparent spheres, called depth sphere, which varies in size in accordance with the depth at which the cursor is located. Bound bubble cursor has a 2D ring, which varies in size according to the depth of the cursor. In both of these techniques, the object in contact with the inner sphere is the one selected. In depth bubble cursor the object closest to the center of the cursor, among all the objects in contact with the sphere is selected.

Figure 6 (a) The Bubble cursor dynamically resizing its activation area depending on the proximity of surrounding

targets to select one target. (b) Double, bound and depth bubble cursor. Objects are selected in accordance with a

depth sphere, 2D ring, and cursor at center.

Another technique that replaced the 3D mouse cursor for a semi-transparent volume, called Silk Cursor, Zhai et al., 1994. Here, the objects within the volume cursor were selected. With this, the cursor activation area was increased, reducing the acquisition times. The problem with this technique is that multiple objects could be brought under the cursor at a time, hence making it difficult for single

object selection. The click-and-cross technique (Findlater et al., 2010) allows users to select an area of the screen, and expand the items contained in that area into arcs that can be selected by crossing rather than pointing. This is similar to the approach used in SQUAD but requires more precision from the user when many objects are presented on the screen. Tumbler uses a stack representation to show occlusion with different layers, and splatter uses object proxies to create a new view where all objects are accessible. The motion-pointing technique (Fekete et al., 2009) allows users to select individual objects without pointing at them, by assigning different types of elliptical motions to each object. It also reduces the number of selectable objects by selecting the top four motion matches and distributing them in a pie menu for direct selection. This technique, however, may not be suited for interfaces with many objects, as the required precision would increase. Starfish technique [Wonner et al. 2012] employs a starfish-shaped closed surface. Each branch ends exactly on preselected near targets. When the desired target is captured by one of the branches, the user can lock the shape and select the desired target.

In another vector-based pointing technique, image plane selection, Pierce et al., 1997, the ray originates from the user's viewpoint and extends through the virtual hand. With this technique, the user interacts with the 2D projections that 3D objects in the scene make on its screen plane. The image plane technique is shown in figure 7 (a). In the Head Crusher technique of the image plane, the user positions his thumb and forefinger around the desired object in the 2D image. The object is selected by casting a ray into the scene from the user’s eye-point between the user’s forefinger and

9

thumb. Sticky Finger technique provides an easier gesture when picking very large or close objects by using a single outstretched finger to select objects on the user’s image plane.

(a) (b) (c) (d)

Figure 7 (a) A third person’s view of the Head-Crusher technique in which the object is between the user's finger and

thumb in its image plane. (b) Sticky Finger technique uses the outstretched finger gesture to select objects in the

image plane. (c) Zoom technique zooms in the area inside the cursor hence providing the user with a larger area to

aim at. (d) Expand Technique uses zoom and SQUAD to select objects.

Cashion et al., 2012, improved upon Raycasting and SQUAD, and developed two variations

namely, Zoom and Expand (figure 7 (c) and (d)). The Zoom technique is an extension to raycasting that helps select small or partially occluded objects by first zooming in on the region of potential targets. The area inside the cursor is zoomed in to provide ample area for selection, however, loses the original context. The Expand technique is a variation of Zoom and SQUAD that selects with progressive refinement. After zooming in, the selected target objects are presented in a grid for the user to choose from depending on the number of targets selected. Cashion and LaViola, 2014, proposed a new 3D selection technique, Scope (figure 8), which dynamically adapts to the environment by altering the activation area of the cursor and visual appearance with relation to velocity. It adapts the speed dependent behavior of Dynaspot.

Figure 8. Scope selection technique with cursor activation area small and large with slow user input (left) and fast

user input (right) simultaneously.

Haan el al., 2005, IntenSelect, is a novel selection technique that dynamically assists the user

in the selection of 3D objects in VEs. A scoring function is employed to calculate the score of objects, which fall within a conic selection volume. By accumulating these scores for the objects, a dynamic, time-dependent, object ranking is obtained. The highest ranking object is indicated by bending the selection ray towards it and the selection is made. The Hook technique (Ortega 2013) employs a scoring system similar to IntenSelect. The primary difference is that Hook computes the distance from the cursor to each object, and derives a score based on that measurement. This method allows pointing in dense 3D environments and on targets moving with high velocity. These techniques use heuristic methods. A behavioral approach of selection, PRISM (Precise and Rapid Interaction through Scaled Manipulation) was presented by Frees and Kessler, 2005. This method dynamically adjusts the control-display ratio. It scales up hand movement for distant selection and manipulation and scales the hand movement down to increase precision. While these techniques can achieve high levels of precision, they cause a significant mismatch of the physical pointing direction and pointing position, and the mapping is nonlinear. The Smart ray, Grossman and Balakrishnan, 2006, employs target selection based on target weights to determine which target should be selected when multiple targets are intersected. Target weights are continuously updated based on their proximity to the ray cursor.

10

This technique, however, may not be suited for interfaces with many objects as the required precision would increase.

Figure 9. The user selects a highest ranking object which is an occluded, moving and far-away object using the

bending ray.

In addition to research experiments, a range of commercial HMD based VR platform enable controller

based object selection. HTC Vive uses a platform dependent two motion tracked remote controls to

select and manipulate virtual objects. The remote controls have a button each which casts a ray in the

VE and further get selected when pressed. Similar to HTC Vive, Oculus Rift also uses a platform

dependent Oculus touch hand controllers that cast a ray to hover on the virtual objects. The objects

are selected when a button on any of the Oculus touch is pressed. Google Daydream viewer and

Samsung VR platform use a controller supported through a ray casting which gets selected when a

button is pressed on the controller. Although all commercially available platform performs object

selection through ray casting, no evidence is present for exploring these controllers for object

selection in a dense and occluded dense VE.

4.1.1. Research gap in controller based object selection method

Overall, a wide variety of controller-based input techniques has been explored for object

selection in 3D UI and VR interfaces. Although in past few years, the approaches have advanced from

simple ray-casting based methods and its variations to disambiguation and heuristic based input

methods, they still possess many challenges that impact effective object selection in VR interfaces.

The current research on controller-based input method for object selection is limited in investigating

them for dense environment and objects which are partially or fully occluded. While some techniques

do exist, there has been less exploration of how these techniques are affected by the environment

density and target visibility. Moreover, most of these techniques are explored in a desktop and

projection based VR, but limited in HMD VR interfaces. Exploration of suitable objection selection

techniques in HMD VR interfaces is utmost importance due to (a) its increased usage and application

in recent years and (b) different challenges than desktop and projection based VR interfaces. At last,

which in our opinion is the most important one - all these techniques require an active controller for

object selection. This demands a user to hold an active object which is not suitable in conditions of

physical and virtual multi-tasking (e.g. object selection in a VE while performing physical activities in a

real world) as the user needs to keep shifting between physical objects and the controllers. Moreover,

it is unnatural, demands to learn to hold different devices and input interactions designed for different

technology platform (e.g. hand gloves, remote control, oculus touch etc.) which often increases the

cognitive load and learning curve among the targeted users.

This demands for new approaches that overcome the challenges of consistently learning to

hold a physical object, learn to use new input interactions and take factors of different object density,

object distance, object size and object occlusion in a VE. The following section presents the literature

on controller-less object selection method, its advantages and the scope of future research.

11

4.2. Controller-less input techniques for object selection

In recent years, researchers have explored controller-less input interaction methods extensively due to the challenges experienced while holding a physical controller for controller supported object selection in VR interfaces. Most common of them is body-gesture based input interactions, including upper and lower body gestures. This section covers the literature of controller-less input methods, advantages and research gaps in object selection.

Similar to controller-based input interaction method, ray casting methods have also been explored for controller-less input interaction for object selection. Mayer et al., 2018 proposed 4 ray casting pointing techniques in which the ray is casted using index finger, head, eye-finger, and forearm. This is achieved using the position and orientation of the marker position and additional measurements. They investigate the possible use of freehand midair pointing in the real and virtual environment and further extended existing correction models to investigate the impact of visual feedback on humans’ pointing performance. Similarly, Matulic and Vogel 2018, explored Multiray (multi finger raycasting) where each finger directly and independently emits an unmodified ray from a distance on large scale displays. The proposed techniques, created by hand postures form 2D geometric shapes to trigger actions and perform direct manipulations that extend the single-point selection. The multiray technique is shown in figure 10.

Figure 10. Shapes detected when aligning multi-finger raycast projections.

Controller-less ray casting technique is also used for object selection using two hands. Wyss et al. 2006 developed a technique called iSith. It uses two ray pointers, and a 3D point to select a target when the distance of these two rays falls below a defined threshold. Although this method is useful to select occluded objects, it occupies both hands while selecting the virtual objects. The use of both hands may not be suitable for application that requires multi-tasking (e.g. selection & manipulation of virtual objects). The iSith technique is shown in figure 12(a).

Another common method alternative to ray casting in controller-less input interaction is the hand extension metaphor. In this approach, the user's hand position is tracked in real space, typically using cameras, and the hand position is used to control a cursor or a hand avatar in 3D. The area of influence is the volume defined by the virtual hand, the control-display ratio is 1:1, the motor and visual space are coupled, no automatic process are provided and the feedback provided is the visualization of the 3D virtual hand avatar. Hand extension method is shown in figure 11.

Figure 11 (a) Hand extension metaphor employs a hand avatar in 3D for grabbing objects.

An extension of hand extension metaphor is presented through the method of arm-length. Arm-length refers to techniques where the length of the users’ arm limits the reach of the virtual objects. They are classified under the scaled technique. Another example of this technique is the Go-

12

Go immersive interaction technique, Poupyrev et al., 1996, which uses the metaphor of interactively growing the user’s arm and non-linear mapping for reaching and manipulating distant objects. This technique allows for seamless direct manipulation of both nearby objects and those at a distance. The Stretch Go-Go technique improves on Go-Go, by being able to extend the virtual arm until infinite. HOMER (Hand-Centered Object Manipulation Extending Ray-casting) technique in the most basic technique in which, the user selects the object with the ray, but instead of the object becoming attached to the light ray, the virtual hand moves to the object position and the object is attached to the hand. When the object is dropped, the hand returns to its natural position. This allows simple grabbing and manipulation. The Scaled HOMER improves upon the Go-Go technique by increasing precision when interacting with objects. Grossman and Balakrishnan 2006, investigated pointing to a single target with the hand extension metaphor in 3D volumetric environments with respect to the height, width and depth of targets and moving angle. They found that moving forward and backward to select targets was significantly slower than moving in other directions.

Researchers have also investigated multi-finger gestures in object selection for VR interfaces. Mendes et al. 2014, proposed and evaluated a set of techniques using a pinch (using index finger and thumb) and grab gestures to directly select and manipulate objects in mid-air using a stereoscopic table top. Similarly, Vogel and Balakrishnan 2005 also explored different alternatives for triggering selection for free-hand pointing in large displays without using any button. They proposed two different hand gestures to perform the selection, AirTap and Thumb Trigger in a combination of visual and auditory feedback. Air Tap and Thumb Trigger are shown in figure 12(b).

(a) (b)

Figure 12 (a) Selection of occluded objects using Isith technique (b) AirTap is a “down and up” gesture of the index

finger. Thumb Trigger is an “in and out” gesture of the thumb.

Lin et al., 2016 implemented three ways of interacting with objects: innate pinching, magnetic force, and a physical button attached to the index finger. Innate pinching requires grasping an object with a pinch gesture and in magnetic force, a finger magnetically attracts an object within the grabbing object distance to grab an object. This implementation solves the accuracy problem other pinching implementation presents. However, it also requires more concentration to select and deselect objects. It is also often hard to correctly select objects which are close together. Figure 13 (a) and (b) represents selection through magnetic force and innate pinch respectively.

Bowman and Wingrave, 2001; Bowman et al., 2002 initially explored Menu selection using Pinch gestures. Ni et al. 2011 introduced rapMenu technique, which allows menu selection by controlling wrist tilt and employing multiple pinch gestures. It takes advantage of the multiple discrete gesture inputs to reduce the required precision of the user hand movements.

13

(a) (b)

Figure 13 (a) selecting an object using magnetic force moves the object to hand. (b) selecting an object using innate

pinch using pinch gesture

Researchers have explored disambiguation methods for object selection to overcome challenges of dense VEs. Ren and O’Neill, 2013 investigated the design of freehand 3D gestural interaction that could be used in everyday applications without complex configuration or attached devices or markers. The study investigates target selection techniques in dense and occluded 3D environments. The selection is accomplished by using a selection cone whose apex is kept fixed while the center of the cone base can be moved in a vertical plane by the movement of the user’s hand. The objects that are intersecting the cone can be selected, and a menu based selection disambiguation method similar to SQUAD is employed. In this method, the user can perform a hand pull gesture in order to display a 2D menu that lists the objects being intersected by the selection cone. Here, the selection is accomplished by picking one of the listed items. This technique is shown in figure 14. While such menu based disambiguation methods are accurate, they remove the environmental context from the selection procedure and reduce the user’s sense of presence in the virtual environment. Moreover, these techniques do not investigate virtual environments for occluded objects.

Figure 14. (a) Menu cone selects the intended target (in yellow) along with other targets. (b) A pull gesture confirms

the selection and a menu appears. (c) A directional gesture is performed to select the target.

Vosinakis and Koutsabasis 2018, evaluated grasping (using all fingers) with bare hands by providing visual feedback techniques. They presented the findings in terms of design recommendations for bare hands interactions that involve grasp-and-release. Mendes et.al., 2017, proposed PRECIOUS, a novel mid-air technique for selecting out-of-reach objects featuring iterative refinement in VR interfaces. PRECIOUS (Progressive REfinement using cone-casting in Immersive virtual environments for Out-of-reach object Selection) uses cone-casting from the hand to select multiple objects and moves the user closer to the objects in each refinement step, to allow accurate selection of the desired target. It offers infinite reach, using an egocentric virtual pointer metaphor. While pointing, users can also make the cone aperture wider or smaller, and change the cone’s reach. While it is a good technique for selecting objects at a distant, it might be difficult to manipulate the selection cone in such a way that it only intersects a single object when two objects are very close to each or for occluded objects in cluttered environments. The method of object selection in PRECIOUS is shown in figure 15.

14

Figure 15 (a) Selection cone intersecting various objects. (b) refinement phase, moving the user to objects (c) single

object selection. (d) returning to an original position with the object selected.

Moore et al., 2018, proposed a novel technique based on voting. Vote oriented technique enhancement (VOTE) for 3D selection votes for the indicated object during each interaction frame and then selects the object with the most votes during confirmation. IDS method, Periverzov and Ilies, 2015, offers hand placement fault tolerance according to the level of confidence shown by users with respect to the position in space of their hands. A proximity sphere is placed around the simplified hand model of the user such that the fully extended fingers of the hand touch the interior surface of the sphere. The proximity sphere is swept along the path by the motion of the hand, and the objects that are intersected by it are considered to be candidate objects for selection. The size of the proximity sphere is adjusted according to the users’ level of confidence about the position of their hand. This method is shown in figure 16.

Figure 16. IDS method adopts a proximity sphere along the path described by the motion of the

hand, and the objects that are intersected by it are considered for selection.

In addition to hand, arm and finger-based gestures, researchers have investigated head

movement for object selection in VR interfaces. Ramcharitar and Teather, 2018, presented EZ cursorVR (figure 17) which is a 2D head coupled cursor fixed in the screen plane for selection in HMD VR. The control cursor was fixed in the center of the field of view, and thus can be controlled by the user’s head gaze as well as by external input devices.

Figure 17. Movement of EZCursor VR using Head-movement and/or external input device.

In addition to body gestures, numerous researchers have investigated multimodal interactions for object selection, such as holding a device in the air, voice or by tracking the users’ limbs or eyes. For example, Bolt 1980, coupled gesture tracking with voice commands to allow interactions such as ‘‘put that there’’, and eye-gaze tracking for window selections. Pierce et al., 1997 describe a variety of image plane interaction techniques that allow users to interact with 2D projections of 3D spaces using head-tracked immersive virtual environments. Similarly, commercially available google cardboard

15

(Yoo and Parker, 2015) or similar low-cost cardboard platform uses ray casting method that explores the VE using the head movements and activated using dwell gaze. Other systems have coupled hand and body gestures to enhance expressiveness and playfulness (Brewster et al., 2003; Krueger, 1991). Lee et al., 2003, compared image-plane selection to a hand-directed ray, head-directed ray, and a ray controlled by both the head and hand, and report that image-plane selection performed best. Although researchers have investigated the use of body-gestures combined with voice or other activation methods, they still face challenges of different languages, dialect, privacy issues (especially in a public environment) and accuracy issues in a crowded setup.

4.2.1. Research gap in controller-less methods for object selection

The literature suggests extensive use of body-gestures for controller-less object selection in

VR interfaces. This includes the use of ray casting including multi-finger and two hand ray casting,

virtual hand based object selection, arm and finger-based gestures other than pointing and

disambiguation techniques. Most of the techniques are studied in desktop and projection based VEs,

limited work is found for object selection in HMD based VR interfaces. Moreover, very few techniques

have been designed for object selection in dense target VEs, or the selection of objects which are

occluded from the user’s viewpoint. Further, the issues of object distance, object proximity, and object

size is yet untouched in the literature of object selection using controller-less input method. It is also

worth noting that none of these techniques are adopted through a user-centered design approached,

but forced to the users through a developer-driven object selection techniques.

Overall, body gestures, in general, have been found one of the most natural input methods in

HCI [references], however, exploration of user-centric body-gestures for object selection in HMD

based VR interfaces have not been sufficiently explored. Moreover, the developer-driven body-

gestures in object selection, issues in dense and occluded dense VE, and impact of object proximity,

object distance and object size in an effective object selection still remain a challenge and demands

careful HCI interventions. This research aims to address this gap and study the use of user-centric

controller-less body gestures in object selection. The experiments will focus on the technology

platform that supports HMD based VR interfaces due to its evident research gap in the study of

controller-less object selection. The recent growth in commercial HMD VR devices and its extensive

application areas makes it a timely choice, whereas different challenges to traditional desktop and

project-based VR interface provides an opportunity for novel contribution.

16

5. Research questions

5.1. RQ 1: Which of the new controller-less body-gestures are required for effective object

selection in the dense VE for HMD based VR?

5.1.1. How does the proximity of virtual objects influence the effectiveness of designed

controller-less body-gestures?

5.1.2. How does the size of the virtual objects influence the effectiveness of the designed

controller-less body-gestures?

5.1.3. How does the distance of the virtual objects from the users’ view influence the

effectiveness of the designed controller-less body-gestures?

5.2. RQ 2: Which of the new controller-less body-gestures are required for effective object

selection in an occluded dense VE for HMD based VR?

5.2.1. How does the proximity of virtual objects influence the effectiveness of designed

controller-less body-gestures?

5.2.2. How does the size of the virtual objects influence the effectiveness of the designed

controller-less body-gestures?

5.2.3. How does the distance of the virtual objects from the users’ view influence the

accuracy and task completion time of designed controller-less body-gestures?

17

6. Methodology to design experiments for presented research questions

The research aims to investigate user-centric body-gesture based object selection method for

dense and occluded dense VE. Moreover, it will study the influence of object proximity, object size

and object distance on each of the designed selection methods. To investigate our RQs, a series of

experiments will be conducted over the next 18 months. 2 experiments will be conducted to extract

user-centric controller-less body-gestures for object selection, followed by 6 studies to compare its

effectiveness to commonly used objection selection methods identified from the literature. Finally,

we will present a series of design guidelines or framework as a core contribution of this thesis. The

methodology is explained briefly in the following sections.

First, two user-centered gesture design studies will be conducted to investigate user-centric

body-gestures in dense and occluded VE. Moreover, it will also study body-gestures which for different

object proximity, object distance and object sizes. The body-gestures will be elicited majorly based on

the gesture frequency, appropriateness to each task and ergonomic considerations. This study will

help identify user-centric body-gestures suitable to different VE for HMD based VR interfaces, which

will be further compared to most commonly used body-gestures for object selection.

This will be followed by 2*3 further experiments comparing the newly designed user-centric

body gestures to most commonly used body-gestures identified from the literature. The experiments

will study the effectiveness – mainly error rates or accuracy of object selection, task completion time,

user preferences and ease of use. Finally, based on these results, a series of design guidelines or a

framework design will be proposed as a core contribution to this thesis. The guidelines or framework

will enable practitioners and/or researchers to design controller-less body gestures for object

selection in a dense and occluded dense VE with varied object parameters.

18

7. References

1. Bolt, R.A., 1980. Put-that-there: voice and gesture at the graphics interface. In: Proceedings

of the Seventh Annual Conference on Computer Graphics and Interactive Techniques, Seattle,

Washington, United States, pp. 262–270.

2. Bowman, D. A., & Hodges, L. F. (1997, April). An evaluation of techniques for grabbing and

manipulating remote objects in immersive virtual environments. In Proceedings of the 1997

symposium on Interactive 3D graphics (pp. 35-ff). ACM.

3. Poupyrev, I., Billinghurst, M., Weghorst, S., & Ichikawa, T. (1996, November). The go-go

interaction technique: non-linear mapping for direct manipulation in VR. In Proceedings of the

9th annual ACM symposium on User interface software and technology (pp. 79-80). ACM.

4. Fabio Buttussi and Luca Chittaro. 2018. Effects of different types of virtual reality display on

presence and learning in a safety training scenario. In IEEE Transactions on Visualization and

Computer Graphics, 24(2), 1063-1076.

5. Cockburn, A., & Firth, A. (2004). Improving the acquisition of small targets. In People and

Computers XVII—Designing for Society (pp. 181-196). Springer, London.

6. Kopper, R., Bacim, F., Bowman, D., 2011. Rapid and accurate 3D selection by progressive

refinement. In: Proceedings of the IEEE Symposium on 3D User Interfaces, pp.67–74.

7. Rosa, D. A. W., & Nagel, H. H. (2010, November). Selection techniques for dense and occluded

virtual 3d environments, supported by depth feedback: Double, bound and depth bubble

cursors. In Chilean Computer Science Society (SCCC), 2010 XXIX International Conference of

the (pp. 218-225). IEEE.

8. Chapuis, O., Labrune, J. B., & Pietriga, E. (2009, April). DynaSpot: speed-dependent area

cursor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp.

1391-1400). ACM.

9. Bowman, D., Wingrave, C., Campbell, J., & Ly, V. (2001). Using pinch gloves (tm) for both

natural and abstract interaction techniques in virtual environments.

10. Forsberg, A., Herndon, K., & Zeleznik, R. (1996, November). Aperture based selection for

immersive virtual environments. In Proceedings of the 9th annual ACM symposium on User

interface software and technology (pp. 95-96). ACM.

11. Pierce, J.S., Forsberg, A.S., Conway, M.J., Hong, S., Zeleznik, R.C., Mine, M.R., 1997. Image

plane interaction techniques in 3D immersive environments. In: Proceedings of the 1997

Symposium on Interactive 3D graphics, Providence, Rhode Island, United States, ACM, pp. 39–

43

12. Daniel Vogel and Ravin Balakrishnan. Distant freehand pointing and clicking on very large, high

resolution displays. In UIST ’05: Proceedings of the 18th annual ACM symposium on User

interface software and technology, pages 33–42. ACM Press, 2005.

13. Grossman, T., & Balakrishnan, R. (2004, April). Pointing at trivariate targets in 3D

environments. In Proceedings of the SIGCHI conference on Human factors in computing

systems(pp. 447-454). ACM.

14. Zhang Hui. 2017. “Head-mounted display-based intuitive virtual reality training system for the

mining industry". International Journal of Mining Science and Technology

15. Bowman, D., Wingrave, C., 2001. Design and evaluation of menu systems for immersive virtual

environments. In: Proc. VR 2001, pp. 149–156.

16. Bowman, D., Wingrave, C., Campbell, J., Ly, V., Rhoton, C., 2002. Novel uses of pinch gloves

for virtual environment interaction techniques. Virtual Reality 6 (3), 122–129

19

17. Cashion, J., Wingrave, C., & LaViola Jr, J. J. (2012). Dense and dynamic 3d selection for game-

based virtual environments. IEEE transactions on visualization and computer graphics, 18(4),

634-642.

18. Changyang Li, Wei Liang, Chris Quigley, Yibiao Zhao, Lap-Fai Yu. 2017. “Earthquake safety training through virtual drills". IEEE Transactions on Visualization and Computer Graphics, 23(4), 1275-1284

19. Rosa, D. A. W., & Nagel, H. H. (2010, November). Selection techniques for dense and occluded

virtual 3d environments, supported by depth feedback: Double, bound and depth bubble

cursors. In Chilean Computer Science Society (SCCC), 2010 XXIX International Conference of

the (pp. 218-225). IEEE.

20. Bolt, R. A. (1980). “Put-that-there”: Voice and gesture at the graphics interface (Vol. 14, No. 3,

pp. 262-270). ACM.

21. Tassos A. Mikropoulos and Antonis Natsis. 2011. Educational virtual environments: A ten year

review of empirical research (1999-2009). Computers and Education, 56(3), 769-780.

22. Bowman, D. A., Johnson, D. B., & Hodges, L. F. (2001). Testbed evaluation of virtual

environment interaction techniques. Presence: Teleoperators & Virtual Environments, 10(1),

75-95.

23. Lee, S., Seo, J., Kim, G. J., & Park, C. M. 2003. Evaluation of pointing techniques for ray casting selection in virtual environments. In the International Conference on Virtual Reality and its Application in Industry (Vol. 4756, No. 1), 38-44.

24. Andrew Y. C. Nee and Soh K. Ong. 2013. Virtual and augmented reality applications in

manufacturing. In 7th IFAC Conference on Manufacturing Modelling, Management and

Control, 15-26

25. De Haan, G., Koutek, M., & Post, F. H. (2005, October). IntenSelect: Using Dynamic Object

Rating for Assisting 3D Object Selection. In IPT/EGVE (pp. 201-209).

26. Liang, J. and Green, M. (1994). JDCAD: A highly interactive 3D modeling system. Computers

and Graphics. 18(4): p. 499-506.

27. Ni T, Bowman DA, North C, McMahan RP (2011) Design and evaluation of freehand menu

selection interfaces using tilt and pinch gestures. Int J Hum Comput Stud 69(9):551–562

28. Cashion, J., & LaViola, J. J. (2014, March). Poster: Dynamic adaptation of 3D selection

techniques for suitability across diverse scenarios. In 3D User Interfaces (3DUI), 2014 IEEE

Symposium on (pp. 165-166). IEEE.

29. Shailey Minocha, Ana-Despina Tudor, Steve Tilling. 2017. Affordances of mobile virtual reality

and their role in learning and teaching. In Proceedings of the 31st British Computer Society

Human Computer Interaction Conference (p. 44). BCS Learning & Development Ltd.

30. David Lockwood. 2004. Evaluation of Virtual Reality In Africa: An educational perspective; Retrieved September 8, 2018 from http://unesdoc.unesco.org/images/0013/001346/134607e.pdf

31. Fatima Gutierrez, Jennifer Pierce, Victor Vergara, Robert Coulter, Linda Saland, Thomas P. Caudell, Timothy E. Goldsmith, and Dale C. Alverson. 2007. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education in Medicine Meets Virtual Reality 15: In Vivo, In Vitro, In Silico: Designing the Next in Medicine 125 (2007), 155

32. Tassos A. Mikropoulos. 2006. Presence: a unique characteristic in educational virtual environments. Virtual Reality 10, 3-4 (2006), 197–206.

33. Cobb, S. V., Nichols, S., Ramsey, A., & Wilson, J. R. (1999). Virtual reality-induced symptoms and effects (VRISE). Presence: Teleoperators & Virtual Environments, 8(2), 169-186.

20

34. Nick Yee and Jeremy Bailenson. 2007. The Proteus effect: The effect of transformed self-representation on behavior. Human Communication Research 33, (2007), 271–290.

35. Nick Yee, Jeremy N. Bailenson, Nicolas Ducheneaut. 2009. The Proteus effect: Implications of transformed digital self-representation on online and offline behavior. Communication Research, 36(2), 285-312

36. Adi Robertson. 2017. Google has shipped over 10 million Cardboard VR headsets. Retrieved April 14, 2018 from https://www.theverge.com/2017/2/28/14767902/googl-cardboard-10-million-shipped-vr-ar-apps

37. Theodore Svoronos, Deusdedit Mjungu, Prabhjot Dhadialla, Rowena Luk, Cory Zue, Jonathan Jackson, Neal Lesh. 2010. CommCare: Automated quality improvement to strengthen community-based health. Weston: D-Tree International.

38. Danilo D. O. Robertson. 2017. Virtual reality training in Southern Africa. Retrieved April 14, 2018 from https://www.unido.org/stories/virtual-reality- training-southern-africa#story-start

39. Argelaguet Sanz, F. (2011). Pointing facilitation techniques for 3d object selection on virtual environments. Universitat Politècnica de Catalunya.

40. A. Worden, N. Walker, K. Bharat, and S. Hudson. Making computers easier for older adults to use: area cursors and sticky icons. In Proc. CHI ’97, 266–271. ACM,1997.

41. Hasan, K., Grossman, T., & Irani, P. (2011, May). Comet and target ghost: techniques for selecting moving targets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 839-848). ACM.

42. Yoo, S., & Parker, C. (2015, August). Controller-less interaction methods for Google cardboard. In Proceedings of the 3rd ACM Symposium on Spatial User Interaction (pp. 127-127). ACM.

43. Cockburn, A., & Firth, A. (2004). Improving the acquisition of small targets. In People and Computers XVII—Designing for Society (pp. 181-196). Springer, London.

44. Frees, S., & Kessler, G. D. (2005, March). Precise and rapid interaction through scaled manipulation in immersive virtual environments. In Virtual Reality, 2005. Proceedings. VR 2005. IEEE (pp. 99-106). IEEE.

45. Findlater, L., Jansen, A., Shinohara, K., Dixon, M., Kamb, P., Rakita, J., & Wobbrock, J. O. (2010, October). Enhanced area cursors: reducing fine pointing demands for people with motor impairments. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (pp. 153-162). ACM.

46. Vanacken, L., Grossman, T., & Coninx, K. (2007, March). Exploring the effects of environment density and target visibility on object selection in 3D virtual environments. In null(p. null). IEEE.

47. Mine, M. (1995). Virtual environment interaction techniques, UNC Chapel Hill Computer Science Technical Report TR95-018.

48. Chatterjee, I., Xiao, R., & Harrison, C. (2015, November). Gaze+ gesture: Expressive, precise and targeted free-space interactions. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (pp. 131-138). ACM.

49. Ren, G., & O'Neill, E. (2013). 3D selection with freehand gesture. Computers & Graphics, 37(3), 101-120.

50. Brewster, S., Lumsden, J., Bell, M., Hall, M., Tasker, S., 2003. Multimodal ‘eyes-free’ interaction techniques for wearable devices. In: Proceedings of the CHI’03: Conference on Human Factors in Computing Systems, Ft. Lauderdale, Florida, ACM, pp. 473–480.

51. Krueger, M., 1991. VideoPlace and the interface of the future. In: B., Laurel (Ed.), The Art of Human Computer Interface. Addison Wesley, pp. 417–422

52. Lucas, J. F. (2005). Design and evaluation of 3D multiple object selection techniques (Doctoral dissertation, Virginia Tech).

21

53. Teather, R. J. 2018. EZCursorVR: 2D Selection with Virtual Reality Head-Mounted Displays. environments, 4, 31.

54. Periverzov, F., & Ilies, H. (2015, March). IDS: the intent driven selection method for natural user interfaces. In 2015 IEEE symposium on 3D user interfaces (3DUI) (pp. 121-128). IEEE.

55. Moore, A. G., Hatch, J. G., Kuehl, S., & McMahan, R. P. (2018). VOTE: A ray-casting study of vote-oriented technique enhancements. International Journal of Human-Computer Studies, 120, 36-48.

56. Mendes, D., Medeiros, D., Sousa, M., Cordeiro, E., Ferreira, A., & Jorge, J. A. (2017). Design and evaluation of a novel out-of-reach selection technique for VR using iterative refinement. Computers & Graphics, 67, 95-102.

57. Vosinakis, S., & Koutsabasis, P. (2018). Evaluation of visual feedback techniques for virtual grasping with bare hands using Leap Motion and Oculus Rift. Virtual Reality, 22(1), 47-62.

58. Lin, J., & Schulze, J. P. (2016). Towards Naturally Grabbing and Moving Objects in VR. Electronic Imaging, 2016(4), 1-6.

59. Grossman, T., & Balakrishnan, R. (2006, October). The design and evaluation of selection techniques for 3D volumetric displays. In Proceedings of the 19th annual ACM symposium on User interface software and technology (pp. 3-12). ACM.

60. Zhai S, Buxton W, Milgram P. 1994. The “Silk Cursor”: investigating transparency for 3D target acquisition. In: CHI '94: proceedings of the SIGCHI conference on human factors in computing systems; 1994. p. 459–64.

61. Matulic, F., & Vogel, D. (2018, April). Multiray: Multi-Finger Raycasting for Large Displays. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (p. 245). ACM.

62. Mayer, S., Schwind, V., Schweigert, R., & Henze, N. (2018, April). The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (p. 653). ACM.

63. Wyss, H. P., Blach, R., & Bues, M. (2006, March). iSith-Intersection-based spatial interaction for two hands. In 3D User Interfaces, 2006. 3DUI 2006. IEEE Symposium on (pp. 59-61). IEEE.

64. Ortega, M. (2013, March). Hook: Heuristics for selecting 3D moving objects in dense target environments. In 3D User Interfaces (3DUI), 2013 IEEE Symposium on (pp. 119-122). IEEE.

65. Wonner, J., Grosjean, J., Capobianco, A., & Bechmann, D. (2012, December). Starfish: a selection technique for dense virtual environments. In Proceedings of the 18th ACM symposium on Virtual reality software and technology (pp. 101-104). ACM.

66. Fekete, J.D., Elmqvist, N., Guiard, Y., 2009. Motion-pointing: target selection using elliptical motions. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp. 289–298.

67. Findlater, L., Jansen, A., Shinohara, K., Dixon, M., Kamb, P., Rakita, J., Wobbrock, J.O., 2010. Enhanced area cursors: reducing fine pointing demands for people with motor impairments. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, pp. 153–162

68. Steinicke, F., Ropinski, T., & Hinrichs, K. (2006). Object selection in virtual environments using an improved virtual pointer metaphor. In Computer Vision and Graphics (pp. 320-326). Springer, Dordrecht.

69. Steed, A. and Parker, C. (2004). 3D Selection Strategies for Head Tracked and Non-Head Tracked Operation of Spatially Immersive Displays. 8th International Immersive Projection Technology Workshop.

70. Feiner, A. O. S. (2003). The flexible pointer: An interaction technique for selection in augmented and virtual reality. In Proc. UIST'03 (pp. 81-82).

71. Argelaguet, F., & Andujar, C. (2008, August). Improving 3D selection in VEs through expanding targets and forced disocclusion. In International Symposium on Smart Graphics(pp. 45-57). Springer, Berlin, Heidelberg.

22

72. Alan Wexelblat, Virtual Reality: Applications and Explorations 73. D. A. Bowman, E. Kruijff, J. J. L. Jr., and I. Poupyrev. 2004. 3D User Interfaces: Theory and

Practice. Addison-Wesley 74. Jason Jerald, Sep-2015 The VR Book: Human-Centered Design for Virtual Reality. Morgan &

Claypool

75. Merriam-Webster, 2018. Retrieved Nov 6, 2018, from https://www.merriam-webster.com/dictionary/virtual%20reality