Interaction techniques in Virtual Environments
Interaction techniques in Virtual Environments
[pic]
1998
R. van de Pol
[pic]
Department Computer Graphics & CAD/CAM
Faculty of Technical Mathematics and Informatics
Delft University of Technology
[pic]
Graphics, Visualization and Usability Center
Georgia Institute of Technology
Preface
This report has been written as a research assignment, which is a required part of the study Technical Informatics of the Delft University of Technology. It is a literature study on interaction techniques in Virtual Environments (VEs) that serves as a basis for my Master’s Thesis project. I will do this project at the Graphics, Visualization and Usability (GVU) Center of the Georgia Institute of Technology. The project involves the research of interaction techniques suitable for the Virtual Workbench, a relatively new interface paradigm in the field of Virtual Environments. The Virtual Workbench will primarily be used for the development applications for fast, exploratory visualization of large data, including terrain data and atmospherical data.
My advisors for this project are ir. F.H. Post (TU Delft) and W. Ribarksy, Ph.D. (GVU Center).
R. van de Pol
March 1998
Summary
For virtual environments to be effective, users should be able to perform useful tasks in them. This implies that the user should be able to interact with the environment and the objects within it. The ability of the user to maintain a sense of ‘presence’ in the environment is often seen to be the main goal of virtual environment systems. One way to achieve this is to try to mimic real-world interaction tasks in virtual environments by allowing a person to use natural and intuitive gestures and movements. However, this is often too restricting. Therefore, in order to improve task performance, it is useful to extend interaction tasks in virtual environments and enhance them in ways that are not possible in the real world.
A lot of problems exist when interacting with virtual objects and environments. The most important ones are the lack of constraints (such as solid objects and gravity), and technological limitations, which tend to make working in three dimensions very hard. It is important that a unifying framework is developed for interacting in virtual environments, comparable to the WIMP (Windows, Icons, Menus, Pointer) interface in traditional workstation environments. However, because of the very different demands applications can pose on the tasks that should be available to the user, this is very difficult.
This report discusses various interaction techniques that have been developed to perform three basic interaction tasks: navigation, selection, and manipulation. Furthermore, the use of (virtual) menus in virtual environments is investigated. Menus have greatly proven their use in traditional computing environments, and therefore it would seem worthwhile to see if and how they can be useful in virtual environments.
Table of Contents
Preface i
Summary ii
1 Introduction 1
1.1 What are virtual environments? 1
1.1.1 Types of VE systems 2
1.1.2 General tasks in VE systems 3
1.2 Hardware devices for VE systems 4
1.2.1 Output devices 4
1.2.2 Input devices 6
1.3 Why is interaction important? 7
1.4 CHIMP and the Virtual Gorilla Exhibit 8
1.4.1 CHIMP 8
1.4.2 Virtual Gorilla Exhibit 9
2 Interaction in VEs 11
2.1 Presence 11
2.2 Interaction in VEs: why is it hard? 13
2.3 Requirements on interaction in VEs 13
2.4 Basic interaction tasks 16
3 Navigation 17
3.1 Issues of navigation 17
3.1.1 What is navigation? 17
3.1.2 Situation awareness 18
3.1.3 Constraints on navigation 19
3.2 Specifying direction 19
3.2.1 Physical movement 20
3.2.2 Hand directed techniques 21
3.2.3 Gaze directed techniques 22
3.2.4 Object driven techniques 24
3.2.5 Goal driven techniques 25
3.2.6 Dynamic scaling 25
3.2.7 World in miniature (WIM) 25
3.3 Specifying speed 27
3.3.2 Predictor based technique 28
3.4 Navigation in CHIMP and the Virtual Gorilla Exhibit 29
3.4.1 CHIMP 29
3.4.2 Virtual Gorilla Exhibit 30
4 Selection 32
4.1 Issues of selection 32
4.2 Selection techniques 33
4.2.1 Physical intersection 33
4.2.2 Scaled world grab 34
4.2.3 Silk cursor 35
4.2.4 Ray casting techniques 36
4.2.5 Arm extension techniques 37
4.2.6 Image plane techniques 40
4.2.7 Other selection techniques 42
4.3 Selection in CHIMP and the Virtual Gorilla Exhibit 43
4.3.1 CHIMP 43
4.3.2 Virtual Gorilla Exhibit 44
5 Manipulation 45
5.1 Issues of manipulation 45
5.1.1 What is manipulation? 45
5.1.2 Constraints 46
5.1.3 Two-handed manipulation 47
5.2 Manipulation techniques 48
5.2.1 Direct manipulation 49
5.2.2 Tools and widgets 50
5.2.3 Implementing constraints 52
5.3 Manipulation in CHIMP and the Virtual Gorilla Exhibit 53
5.3.1 CHIMP 53
5.3.2 Virtual Gorilla Exhibit 55
6 Virtual menus 57
6.1 Issues of virtual menus 57
6.1.1 What are (virtual) menus? 57
6.1.2 Advantages of virtual menus 59
6.2 Implementation of virtual menus 59
6.2.1 Placement of menus 60
6.2.2 Dimensionality of virtual menus 60
6.2.3 The difficulty of selecting in VEs 62
6.3 Virtual menus in CHIMP and the Virtual Gorilla Exhibit 63
6.3.1 CHIMP 63
7 Discussion and conclusions 66
7.1 Virtual navigation 66
7.2 Virtual selection 67
7.3 Virtual manipulation 67
7.4 Virtual menus 68
7.5 Conclusions 68
8 References 69
1 Introduction
This report aims to give an overview of interaction techniques in virtual environments (VEs). The main focus will be on various software solutions that have been devised to allow the user to easily perform basic tasks in these environments. However, certain hardware issues will also be discussed, since these are often closely related to the software. This chapter provides a (very) brief overview of the field of virtual environment systems (also known as virtual reality). It starts of with a definition of the term virtual environments, after which two types of VE systems will be distinguished. A description of the basic components of a general virtual environment system is included as well. This is followed by a short overview of the various types of hardware devices that are currently being used with VE systems. Next, a classification of the different tasks for which VE systems have been used will be given (with some example applications). After that follows a discussion of the importance of a user being able to interact with the virtual environment. Then, an overview of the following chapters will be given. Finally, two sample applications will be introduced that will used throughout this report to illustrate several of the techniques discussed.
1 What are virtual environments?
A lot of different definitions exist for virtual environments and the virtual reality experience. One that covers the subject fairly well is the following [Barfield ‘95]:
A Virtual Environment (VE) is a computer-simulated world consisting of software representations of real (or imagined) agents, objects and processes; and a human-computer interface for displaying and interacting with these models.
There are a few things worth noting in this definition. First of all, the virtual environment has to be computer-generated. In a lot of texts teleoperator systems, where a robot or vehicle is controlled by a human from a (large) distance, are also categorized as being VE systems. However, these will not be considered in this report. Secondly, ‘displaying’ not only includes stimulation of the visual sense, but of the other senses as well, for example through the use of auditory or tactile displays. Another important thing is that the user should be able to interact with the models (see paragraph 1.3).
1 Types of VE systems
Besides the teleoperator systems mentioned above, two types of VE systems can be distinguished [Dani ‘96]:
• Immersive systems. The user’s field of view is completely surrounded by a synthetic, computer-generated, three-dimensional environment. Usually, the user is wearing some type of head-mounted display (HMD) and the user’s head is tracked to provide him with a perspective-correct view of the scene at all times, depending on the position and orientation of his head. Ultimately, all the sensory information received by the user is generated by a computer, thereby entirely shutting him off from the real world (see Figure 1.1).
• Non-immersive systems. The user typically wears a pair of (see-through) stereo glasses to view a three-dimensional scene on a normal display. With such a system, the user is able to see the real world as well as the computer-generated one. A normal computer monitor can be used as a (visual) display, but more advanced examples of this type of system include the Virtual Workbench and the CAVE system. A picture of the Virtual Workbench is shown in Figure 1.2.
[pic]
Figure 1.1 Immersive system [WebUDW]
[pic]
Figure 1.2 Virtual Workbench [WebSTANFORD]
Both types of systems mentioned consist of three basic components [Barfield ‘95]:
• Computer hardware and software to model and render the environment.
• A logical interface that specifies what parameters of the virtual environment and its models can be changed, and when.
• A physical interface consisting of one or more visual, haptic or auditory displays for presenting the virtual environment to the user, and for monitoring the actions of the user.
In general, the first component should be completely transparent to the user. The logical interface determines the ways in which the user can interact with the VE (e.g. it states that the user can change the position and the orientation of the viewpoint). The physical interface determines how the environment is presented to the user (e.g. by using a head-mounted display). It also determines how the user can perform the interactions as specified by the logical interface (e.g. a 3D tracker is attached to the user’s head to determine the position and orientation of the viewpoint). This report will focus on the logical interface.
2 General tasks in VE systems
Although VE systems have found applications in such diverse fields as engineering, entertainment, science, and training, four general tasks can be distinguished. These will be described and some example application areas will be given for each of them [Barfield ‘95][Vince ‘95].
• Off-line training and rehearsal
• On-line comprehension
• Off-line learning and knowledge acquisition
• On-line performance
In on-line training and rehearsal, the user typically practices a task in the VE system that he has to perform in the real world at a later time. Examples are flight simulators, surgery simulations, and military simulations. Obviously, mistakes can be fatal in these real-world tasks, so the ability to practice them in a virtual world is very convenient.
When on-line comprehension is the goal, the environment is used to reach understanding of and insight into the structure and properties of the data that is displayed or the workings of a particular process. Examples of this include interpretation of scientific data, and simulation of physical and chemical processes. Tasks such as architectural walkthroughs, whose purpose is evaluation of a design, belong to this category as well.
In off-line learning and knowledge acquisition tasks the user utilizes the VE to learn something, and to later apply that knowledge in a different, more abstract form. An example of this is the use of VE systems in education, where particular subjects can be illustrated (one could imagine students of a chemistry class being able to build complex molecules from basic atoms in a VE).
In contrast to the other three tasks, the goal of on-line performance is not to learn something. Instead, the actual actions the user performs are what matters. An example is the manipulation of objects in a remote, or non-viewable environment (such as an undersea robot), but this class also includes VR games.
In the next paragraph a short overview of the hardware devices that can be used to build the (physical) interface for these different types of systems will be given.
2 Hardware devices for VE systems
Various different types of hardware exist to provide user input to the computer, as well as output from the computer to the user. Because of the different (three-dimensional) nature of interaction in virtual environment systems, other devices besides the traditional mouse and keyboard have been developed to make use of these three dimensions for input and output. In this paragraph the general types of these devices will be discussed briefly, together with the way in which they can affect interaction in virtual environments. This list is not meant to be exhaustive, but serves to give an impression of the various techniques that are being used.
1 Output devices
Output devices serve to present the human senses of the user with information from the VE. This paragraph will briefly discuss visual and haptic output devices [DANI ‘96].
1 Visual devices
Visual output devices present the user with a visual image of the environment. These devices can generally be categorized in two classes: they are either immersive or non-immersive (see paragraph 1.1.1). A well known example of an immersive visual display is the head-mounted display (HMD), shown in Figure 1.3. An HMD shuts the user completely off from the real world and contains one
[pic]
Figure 1.3 Head-mounted display [WebVRSYSTEMS]
[pic]
Figure 1.4 Shutterglasses [WebSARA]
screen for each eye on which the graphics are displayed. Often, a HMD contains a pair of headphones for sound as well. In non-immersive displays the user typically wears a special pair of LCD (shutter) glasses (Figure 1.4) for looking at a screen on which the scene is displayed. This screen can be an ordinary monitor, but can also be a Virtual Workbench or a CAVE. To provide a stereo view, the glasses can be synchronized with the screen to alternately present a perspective-correct image to each eye, thus creating the illusion of depth.
Properties of visual output devices that can influence interaction and performance in VEs include image resolution, frame rate, and field-of-view.
2 Haptic devices
As will become clear, a major problem in VEs is the general lack of haptic feedback (the inability of the user to physically feel or touch virtual objects). Although a lot of research is being done is this field (called haptics) the devices that are available are far from perfect and very expensive. Haptic devices provide part of the user’s body (usually a finger or the hand) with touch or force feedback, to give the user the illusion that he is making contact with or holding a virtual object. An example of a haptic device, called the PHANToM, is shown in Figure 1.5. This device allows the user to ‘walk’ the surface of an object with his finger.
An important factor that influences interaction with virtual objects when using haptic devices is the response time of the device (when the user’s finger touches the surface of a virtual object, the haptic device should respond immediately in order for the sensation to be convincing). Other issues are weight, and freedom of movement the device allows.
[pic]
Figure 1.5 PHANToM [WebMITAILAB]
2 Input devices
Input devices allow the user to give commands or perform actions in the virtual environment. This section will discuss tracker devices, various types of gloves, and digitizers (devices that can be seen as extensions of the traditional joystick and mouse).
1 Tracker devices
Trackers are used to keep track of the position and orientation of various parts of the user’s body (usually the head and hands). The workings of these devices can be based on various technologies, including electromechanical, electromagnetic, acoustic, and optical tracking systems. Certain systems also allow the user’s eyes to be tracked [Van Loo ‘97].
Problems of tracking systems are their limited range, tracker noise, and latency which can make precise interaction difficult.
2 Gloves
Datagloves are worn just like normal gloves. They contain hardware that measures the position of the joints of the user’s fingers and can thus recognize gestures. A picture is shown in Figure 1.6. The accuracy of interaction when using these gloves is determined by the ability of the software to accurately distinguish between gestures. A problem may be that it is easy to accidentally make a gesture resulting in an unexpected action in the VE [Dani ‘96].
[pic]
Figure 1.6 Data glove [WebNCSA]
[pic]
Figure 1.7 PinchGloves [WebLOUKA]
PinchGloves are a different type of gloves. Instead of gestures, contact between fingertips is recognized through contact-sensitive pads placed on the glove (see Figure 1.7).
3 Digitizers
Several devices that can be seen as extensions of the traditional joysticks and mice have been developed. These are sometimes called digitizers [Dani '96]. An examples is the Spaceball shown in Figure 1.8 which works like a joystick. In the Spaceball, the stick is replaced by a ball which measures forces applied to it. Another example is the Bat which measures relative displacement in three dimensions (just like a normal mouse does in two dimensions). A wand is a third example (Figure 1.9). This is a hand-held joystick with several buttons. These devices generally allow for very precise interaction.
[pic]
Figure 1.8 Spaceball [WebLOUKA]
[pic]
Figure 1.9 Wand [WebEVL]
Besides the described devices, other types exist as well. Examples include devices to provide the user with 3D spatial sound, to allow voice input, and full body suits [Dani ‘96] [Barfield ‘95].
3 Why is interaction important?
Traditionally, research in virtual environment systems has focused on the visual and (in a somewhat lesser degree) the auditory aspects of these environments. As a result, these two items are well understood and it is now possible to provide the user with a perspective-correct, high-resolution view of a very detailed scene (consisting of hundreds of thousands of polygons), as well as high-quality three-dimensional spatial sound. This has led to some very impressive applications, such as the architectural walkthroughs and the VR games mentioned above.
However, the reason why these applications have been very successful is the low level of interaction that is possible between the user and the VE. It looks great, but that’s it. For most applications, stimulation of only the visual and auditory channels is simply not enough. Just like in the real world, it is very important to provide the user with various means to interact with the virtual objects and the environment itself in order to perform useful tasks. VE systems are basically just a new human-computer interface paradigm. Hopefully it allows the user to interact with computers in a more natural, intuitive (and therefore more effective) way, by letting him use as many of his senses as possible, just as in everyday life. However, because the technology is new and so different from the traditional human-computer interfaces, we are unfamiliar with it and therefore it is important that a lot of research is done to develop general methods that ‘work’, even for inexperienced users [Mine ‘95].
Since the requirements on interaction in virtual environment systems differ considerably from those in traditional (desktop) computing environments, it is important to have a good overview of these requirements, the problems associated with them, as well as possible solutions that have been developed. This report aims to give the user this overview. In the next chapter the problems and requirements associated with interaction will be discussed. In chapters 3, 4, and 5 various methods to implement the basic interactions (navigation, selection, and manipulation) will be described. The subject of chapter 6 will be the use of virtual menus as an interaction tool. The report will conclude with a discussion and some conclusions in chapter 7.
4 CHIMP and the Virtual Gorilla Exhibit
To illustrate several of the interaction techniques, two applications will be examined throughout this paper. The first one is CHIMP (which stands for Chapel Hill Immersive Modeling Program), an immersive system designed to be used for the initial stages of architectural design. It was developed at the University of North Carolina in Chapel Hill [Mine ‘96][WebCHIMP]. The other application is the Virtual Gorilla Exhibit of the Graphics, Visualization, and Usability (GVU) Center at the Georgia Institute of Technology in Atlanta [Allison ‘96][WebGORILLADESIGN]. Although the names may suggest otherwise, the original goals of these applications were very different. They incorporate a lot of different interaction techniques, and form good examples of how useful interaction in virtual environments can be. A description of CHIMP and the Virtual Gorilla Exhibit will be given in this paragraph.
1 CHIMP
As was stated above, CHIMP of the University of North Carolina was developed for the preliminary stages of architectural design. Its main goal has been to take advantage of the fact that the user is immersed, as well as find solutions for the limitations of this. CHIMP can be used to create and manipulate models in an immersive environment. CHIMP uses a head-mounted display, and a magnetic tracking system with three 6 degree-of-freedom sensors (for the head and both hands). The reason for developing CHIMP has been the notion that a person can design three-dimensional spaces more easily in an immersive environment than in a desktop (‘through-the-window’) computing environment., because of the fact that the latter ones are two-dimensional in nature. A picture of CHIMP is shown in figure Figure 1.10.
[pic]
Figure 1.10 CHIMP [WebCHIMP]
2 Virtual Gorilla Exhibit
The GVU Center’s Virtual Gorilla Exhibit was originally developed to allow users to wander around in a virtual replica of Zoo Atlanta’s actual gorilla exhibit. The user ‘becomes’ an adolescent gorilla, and the other virtual gorillas will respond to the user’s actions just like real gorillas would. It is believed that by actually experiencing the way gorillas behave by becoming a gorilla and directly interacting with other gorillas, the user receives a better understanding of gorilla behavior [WebGORILLAS] [Allison ‘96].
This application is being extended to be used as a habitat design and exploration tool as well. The user can explore the gorilla habitat and learn about certain design issues. He is also able to redesign the habitat, for example by adding or moving rocks and trees [WebGORILLADESIGN]. A picture of the Virtual Gorilla Exhibit is shown in Figure 1.11.
[pic]
Figure 1.11 Virtual Gorilla Exhibit [WebGORILLADESIGN]
Both CHIMP and the Virtual Gorilla Exhibit use various ways to allow the user to interact with the virtual environment. They also address some of the problems associated with interaction in virtual environments. These interaction methods and problems will be discussed in the following chapters, as well as the ways in which both applications have approached them.
2 Interaction in VEs
Now that computing power has reached a level that allows for fairly sophisticated interaction between the user and a virtual environment, a lot of research is being done on how people should be able to interact with (objects in) these environments. To perform useful tasks in VEs, the user obviously has to be able to do more than just look around. He has to able to grab objects, inspect them closely, and change some of their parameters, such as their position, orientation, shape, and color. In this chapter, various general issues concerning interaction in VEs will be discussed. First of all, the important concept of presence will be discussed. This is often seen as the central issue in VE systems. Following that is a discussion on what it is that makes interaction in virtual environments so difficult. Next some criteria will be described that can help to improve interaction and task performance in VE systems. Finally, the basic interaction tasks that will be distinguished in this report will be described.
1 Presence
In order for a virtual environment system to be successful, a lot of different requirements can be distinguished. However, it is often stated that the main goal of all virtual environment systems is to allow the user to maintain a sense of ‘presence’ in the VE. The user should feel as if he or is actually ‘there’ inside the virtual environment. The extent of this sense that is necessary for a particular VE system entirely depends on the tasks for which the system is to be used. A definition of (virtual) presence is given in [Barfield ‘95]:
(Virtual) presence is the extent to which human participants in a VE allow themselves to be convinced, while experiencing the effects of a computer-synthesized VE, that they are somewhere other than where they physically are – that ‘somewhere’ being determined by the image, sounds, and physical sensations provided by the computer-synthesized VE to their senses.
Since it is not possible (and often even not desirable) to create an exact replica of the real world, a careful trade-off has to be made for each application to determine which senses should be stimulated for the user to be able to perform his tasks in the VE most effectively. This isn’t a very serious restriction however, because it has been shown that a high sense of presence can occur even when only a subset of the human sensory systems is stimulated. Another thing that is worth mentioning is that it is not clear whether a high level of presence is inherently a good thing. It could very well be that in some cases a high level of presence is distracting to the user, leading to worse task performance [Barfield ‘95]. Unfortunately, presence is a mental state, so it is very difficult to measure. Despite this problem, it is found that the level of presence experienced by a user generally depends on two things [Slater ‘95a]:
• The match between proprioception and the sensory data provided by the VE.
• The match between on one side the various computer-generated displays and on the other side the internal representation systems applied by the user and the mental model he has of himself and the world he lives in.
Proprioception is defined as a person’s sense of the position and orientation of his body and its several parts. Because of this sense, a person ‘knows’ where his left foot is without looking at it, and is able to clap his hands together with his eyes closed [Mine ‘97a][Slater ‘95a]. This can be an important factor in influencing the level of presence experienced in a VE. When the match between proprioceptive cues and sensory data is good, the sense of presence will probably be bigger. For example, when a person moves his leg, this will give him proprioceptive information about the new position and orientation of his leg (for example because of muscles stretching). If the sensory data enhances this feeling (by moving the leg of the virtual body, by creating the sound of object moving through the air), this will lead to a more realistic experience.
The second ‘match’ involves the problem of the same VE leading to different levels of presence with different users. For example, some people are visually dominant, while others are more auditory oriented. So, a VE that stimulates only the visual channel may invoke a high level of presence for the former group, but a much lower level for the last group. Two more specific ways that deal with this match and therefore can enhance the sense of presence are:
• The provision of a virtual body (VB) representation for the user.
• Making sure the virtual environment is ‘place-like’.
A problem of immersive VE systems is that the user is unable to see his own body in the virtual environment. It has been shown that when some sort of representation of the user’s body (a virtual body) is used, the sense of presence can be greatly enhanced [Slater ‘95b][Vince ‘95]. On the other hand, if the user rejects his VB (because it doesn’t match the mental model the user has of himself), this can have a very negative effect on presence [Slater ‘95a][Barfield ‘95].
An environment is ‘place-like’ when it behaves in a predictable and consistent way. To be able to maintain a sense of presence, there should be some form of continuity and stability. If the environment would continuously be changing in a non-predictable way, the user would get confused and disoriented. In other words, the virtual environment should at least have some characteristics of a place in the real world. If this requirement is met sufficiently, it will increase the user’s situation awareness. This means that the user has a general sense of the structure and contents of the environment, as well as his place and the place of other objects within the environment. Situation awareness is very important to effectively perform most tasks [Barfield ‘95].
2 Interaction in VEs: why is it hard?
It is often stated that shortcomings in present day technologies associated with VE systems are the main reason why interaction in these systems is so difficult. The resolution of the display screens in HMDs is too low, the field of view is too small, trackers are inaccurate and have too small a range, there is too much delay to update the scene at an acceptable frame rate. Although this certainly plays a role, there are other major causes as well that won’t disappear just by waiting for technology to improve [Darken ‘93].
A very important problem is the general lack of constraints in VEs [Mine ‘97b]. In the real world, when we perform a task, there are a lot of natural constraints that restrict our movements. In virtual worlds, these constraints are usually not available (or in a very limited form). Also, there are no effective techniques that allow for alphanumeric input by the user (the method usually applied in desktop computing systems) [Mine ‘95]. An example of this is the absence of gravity, which allows a user to move in every direction in the virtual world. This can quickly lead to disorientation. After a while the user may have lost his sense of direction, and has difficulty determining what is up and down, north and south, which can be very frustrating. This problem will be further explored in chapter 5.
Another reason why interaction in VEs is so difficult compared to traditional (desktop) computer systems is the fact that the information, as well as controls to access that information, can be placed anywhere around the user. Therefore, these controls may be hard to find, may obscure the view the user has of the world, or require the user to traverse the world to be able to reach them.
It seems that two of the main reasons why VE systems are being used (direct manipulation and immersion) are also the biggest problems for which solutions have to be found. Because of the novelty of the technology, we are not yet familiar with it. It is important that metaphors for interaction in three-dimensional worlds are developed that are just as effective as the point-and-click metaphor (using a mouse and a mouse pointer to select buttons and menus) in more traditional desktop computer systems [Mine ‘97b].
3 Requirements on interaction in VEs
The requirements that can be posed on interaction in virtual environments mainly serve to eliminate or reduce the problems discussed in the previous paragraph. However, the relative importance of these requirements depends completely on the tasks that are to be performed in the environment. Apart from the obvious requirements on the hardware (high precision, small latency) and the requirements for enhancing presence discussed in paragraph 2.1, several other general requirements can be distinguished for the interaction tasks themselves. If these requirements are met, presence and performance are likely to improve. The following will be discussed in this paragraph [Bowman ‘95]:
• Intuitive interaction
• Constraints
• Affordances
• Mappings
• Feedback
These requirements are valid not only for virtual environments, but for any human-computer interface. However, especially the first two gain extra importance in VEs.
1 Intuitive interaction
First of all, interaction techniques in virtual environments should be intuitive and naturalistic. The basic interactions should be just as easy to perform in a VE as in the real world, and just as easy to learn. This will allow the user to focus on his overall goals without having to think about how to perform a certain basic task [Figueiredo ‘93][Barfield ‘95]. However, interaction usually should not be limited by the real world. The goal is to develop natural forms of interaction and extend them in ways that are not possible in the real world [Mine ‘95]. As an example of task dependence of interaction techniques consider the off-line training and rehearsal tasks discussed in the previous chapter. With this type of tasks it is very important that interaction in the VE closely matches the possible interaction in the real environment in which the real task has to be performed (in a infantry simulation it is for example important that the user has to physically walk to move around in the virtual environment). In an architectural walkthrough however, it might be tedious for the user to have to physically move to get an impression of the design from a particular location. A less natural technique – like teleportation - might be used here that allows the user to quickly inspect the design from several (distant) locations. It is very important that these things are kept in mind when developing a VE system.
2 Constraints
Another very important requirement is the existence of constraints in the VE. This is important because without constraints there are simply too many parameters for the user to modify which leads to confusion. Furthermore, humans are just not very good at precise manipulation. On the other hand, there shouldn’t be so many constraints that the user is unable to perform his task effectively. Usually, input devices provide very few constraints by themselves (a lot of different gestures are possible with data gloves!), so usually these have to be provided in some other form [Bowman ‘95]. There are two possible ways to provide constraints [Mine ‘97b]:
• Virtual constraints
• Physical constraints
When virtual constraints are used, the application simply ignores one or more degrees of freedom of the user’s input. For example, when a user has to specify a three-dimensional tracker is used to position a virtual object on a virtual tabletop, the software makes sure that the object stays on top of the table, despite the fact that the tracker actually moves below the table. Physical controls are actual physical objects to constrain movement, like a real tabletop or a tablet.
It can also be useful to provide the user with a number of different methods to perform a certain task, with each method incorporating more constraints. The user can then choose the method that is most appropriate for the task at hand. When constraints are used correctly, they can greatly improve performance in the VE.
3 Affordances
Affordances are those elements of a tool or object that give away how it should be used or what it can be used for. When a user can instantly figure out the purpose and usage of a tool or object, this will lead to fewer errors and better performance. This is complicated in virtual environments however due to the lack of haptic feedback.
4 Mappings
It is important that good mappings exist between user actions the resulting system actions. The system’s response should be proportional to the user’s action. An example of a good mapping is the one which relates head movements to changes in the viewpoint of the scene. However, higher level mappings are more difficult, mainly because of the large number of degrees of freedom a user has in a VE.
5 Feedback
Feedback keeps the user informed on what the status of the system is . If a user performs a certain action, the system should provide the user with information on the status of this action, especially when it has finished processing the action. When a user is performing a complex task, it is important that he receives a lot of feedback on how he is doing.
4 Basic interaction tasks
Both the number of possible interaction tasks, as well as the ways in which each of them can be implemented are virtually unlimited. However, some basic classifications can be made.
For the implementation of an interaction technique one can choose from the following three methods [Mine ‘95]:
• Direct user interaction
• Physical controls
• Virtual controls
Direct user interaction methods don’t require the user to handle a special tool. Instead they use things like hand tracking, gesture recognition, or gaze direction to determine the intentions of the user. With these methods, intuitive mappings as previously described are very important.
When physical controls are used for interaction tasks, the sense of presence can be greatly enhanced because of the haptic feedback they provide. However, the mappings for these devices are usually not very natural. Another problem may be that the controls are possibly hard to find when the user is wearing a head-mounted display. A good example of a physical control is a steering wheel in a driving simulation. Other examples include joysticks, trackerballs, and buttons.
Virtual controls are very flexible. However, they lack the haptic feedback that is available with physical controls, and they require the user to manipulate a virtual object which is very difficult. However, they can be useful when a lot of different techniques have to be compared, for example when evaluating layout designs of airplane cockpits.
Although the specific interactions that should be available to the user are highly dependent on the task he has to perform in the virtual environment, a few basic interaction tasks can be distinguished that will, in one form or the other, be present in almost all VE systems. In this report navigation, selection, and manipulation will be considered. These basic tasks usually aid the user in performing higher-level, more complex tasks. Therefore, they should be very easy to use and intuitive so that the user can concentrate on performing his actual goals [Poupyrev ‘97]. Navigation deals with ways in which the user can change the position and orientation of the viewpoint in the scene. Selection allows the user to specify one or more objects in which he is interested or with which he wants to do something. Finally, manipulation techniques let the user change certain properties of objects or the scene itself. These interaction tasks will be discussed in the following chapters, together with numerous examples of how they can be implemented.
3 Navigation
In this chapter navigation, the first basic interaction task, will be discussed. The ability to move around in an environment is obviously very important. In the first paragraph a number of issues dealing with navigation in virtual environments will be discussed. After that in paragraph 3.2 and 3.3 several methods that have been used to allow the user to navigate through a virtual environment will be described. Finally, the techniques that have been used in the two sample applications introduced in chapter 1 will be discussed in paragraph 3.4.
1 Issues of navigation
In this paragraph several problems of navigation in virtual environments will be described. To allow the user to successfully navigate through the environment, the VE system must address these problems. First of all, the term navigation as it will be used in this report will be defined. After that, the concept of situation awareness will be explored together with ways to improve the user’s situation awareness. This is very important if the user is to navigate successfully through a (large) environment. As was argued in chapter 2, it is very important to provide constraints in virtual environments. In paragraph 3.1.3, some ways in which constraints can be used to ease navigation will be discussed.
1 What is navigation?
A general definition states that navigation is the process of determining a path to be traveled by any object through any environment [Darken ‘93]. In the context of virtual environments, this usually means changing the position and the orientation of the user’s viewpoint in the environment. To completely specify the user’s movement in the environment, the following two parameters must be specified [Mine ‘95]:
• Direction of motion
• Speed of motion
Navigation usually isn’t a goal in itself in virtual environments. More often it is simply used to get to a different position in the environment in order to perform an other, more important task there. Therefore, navigation methods should be very easy to use and unobtrusive.
In paragraph 3.2, various examples will be given of possible ways to specify the direction of movement. In paragraph 3.3, general methods to specify the speed of motion will be described.
2 Situation awareness
In order for a user to succesfully navigate through a (virtual) environment, it is important that he has a sense of his position in the overall context of the environment. This sense is called situation awareness [Barfield ‘95]. It seems that people build a congnitive map of their environments for use in navigation [Darken ‘93]. People often become disoriented in VEs because these environments usually don’t provide enough cues that allow the user to build a ‘mental map’ of the environment. This is partly caused by the fact that virtual environments contain far less detail than the real world, but another important reason is that the user can’t get a global overview of the environment. However, certain principles from research in real world wayfinding seem to be effective in virtual environments as well [Darken ‘96][Barfield ‘95].
First of all, it has been shown that the use of an electronic map can greatly facilitate search and navigation. When the user is provided with a map, he has an overview of the entire environment at all times. The map can also show the locations of various important goals, landmarks, and objects within the environment, as well as the location and orientation of the user. This will be of great help in maintaining situation awareness. If no map is provided, for most applications it is important that the user is able to easily ‘zoom’ in and out in the environment to switch between an egocentric (first person, immersive) view and a geocentric (third person, panoramic) view [Song ‘93].
Secondly, certain environmental design principles can be applied to help the user build and maintain his mental model of the environment. Examples of this are dividing the large environment into a hierarchy of smaller structures, providing frequent directional cues, including an absolute reference point (e.g. a sun in the sky), and placing landmarks in the scene (e.g. a unique building).
Another reason for users getting disoriented in a VE is that their situation awareness may disappear while they are navigating. An example of this is a navigation technique where the user is instantly ‘teleported’ to a different location in the environment. Although fast, this technique gives the user absolutely no clues about the characteristics of the terrain between the old and the new position. This may result in a complete loss of situation awareness. In this example, it would be better to let the user smoothly fly to the new location, either automatically or under user control. A further improvement here would be to reduce the flying speed when the user is approaching his destination. This will allow him to adjust his mental map to his new surroundings [Barfield ‘95]. It is clear that navigation techniques should be fast and flexible on the one hand, but allow the user to maintain situation awareness on the other.
3 Constraints on navigation
Even if the user has complete control over the direction and speed of movement, he may still get lost in the virtual environment if he has too many degrees of freedom. As was described in paragraph 2.3, it is important to build (physical and virtual) constraints into a virtual environment. One constraint that should generally be present in any virtual environment is that the environment takes up a defined space. The user should not be allowed to navigate out of this space. An example of this is that the user should not be able to navigate through the ‘floor’ of the VE. Furthermore, in most applications it is not necessary for a user to be able to move in any direction. Instead if would be enough if the user can only move over the ground plane. Even if the user has to move in any direction, usually he shouldn’t be able to change his orientation in such a way that he turns ‘upside down’. Another useful constraint would be collision detection so that the user can’t fly through walls and objects [Bowman ‘95].
2 Specifying direction
This paragraph provides an overview of several techniques that have been used to allow somebody to specify the direction in which he wishes to move through a virtual environment. As was stated in paragraph 3.1.1, this is one of the two key parameters that completely specify motion. Except for the workings of each technique, its particular advantages and disadvantages will also be described.
There are several ways to specify the direction of motion [Mine ‘95]:
• Physical movement
• Hand directed
• Gaze directed
• Object driven
• Goal driven
• Dynamic scaling
• World In Miniature (WIM)
Apart from these techniques, it is of course possible to use physical and virtual controls as well, as was discussed in paragraph 2.4. The listed techniques, as well as various examples, will be described in detail in the following paragraphs.
1 Physical movement
Probably the simplest and most natural way for a user to move through a virtual environment is to map movement in the real world, for example walking, into corresponding motion in the VE. This mapping can be one-to-one, but a different mapping is possible as well (e.g. one physical step corresponds to a move of several light years in a visualization of the galaxy) [Mine ‘95].
This technique usually determines changes in the user’s position and orientation through the use of a tracker device that is placed on the user’s head. If the user’s head moves (e.g. when the user takes a step), the viewpoint in the virtual world is updated accordingly.
The advantages of this technique are:
• Ease of use
• Good mapping between proprioceptive cues and sensory data
• Constrained navigation
The main advantage of this technique is its ease of use and its naturalness. It requires no special action from the user, nor the need to handle a special tool. Because of this, the user’s hands are free to perform other tasks. Furthermore, the proprioceptive information the user receives matches the visual information provided by the system (because the user really is walking). This is likely to have a positive effect on the sense of presence of the user. Another advantage may be the inherent constraint of movement on the two-dimensional ground plane. For some applications this is too restricting though. A disadvantage is the slow speed with which the user is able to navigate, which is of course walking speed. However, the technique’s main disadvantage is its limited range. This is caused by the limited range of the tracking system used to monitor the user’s position. Tracker systems that are in use today typically have a range of a few meters. The user obviously has to stay within this range to make sure the system can accurately track his position and orientation. Even if tracker range wouldn’t be the limiting factor, the virtual environment would usually still be larger than the physical environment (the room with the VE system), so usually some other means for navigation has to be available to the user to be able to cover larger (virtual) distances [Mine ‘95].
Several attempts have been made to eliminate this second disadvantage. These techniques usually let the user perform the motions of normal walking without actually changing his positions. It is possible to use a special hardware device for this purpose like a treadmill [Darken ‘97]. An interesting technique is the so-called walking-in-place technique [Slater ‘95a][Slater ‘95b]. This method uses a neural network to determine whether the user is walking in place or not. The user performs the action of walking in place by lifting first one leg and then the other, without actually moving forward. The speed of movement could be determined by the number of ‘steps’ per second. Although this method has the same advantages as actual walking (user’s hands are freed, increased sense of presence) users didn’t find it very effective. Apart from the eliminated problem of limited range, this technique also suffers from the other disadvantages of physical walking. Furthermore, it has been found to easily lead to fatigue when the user has to move around a lot in the VE.
2 Hand directed techniques
With hand directed techniques, the direction of motion is determined by the position and orientation of the user’s hand. Two possible methods are:
• Pointing
• Crosshairs
• Two-handed flying
1 Pointing
When the pointing technique is used, the user points with his or her hand or a hand-held device to the direction in which he or she wants to move (see Figure 3.1). This method is very flexible and intuitive, but it has no inherent constraints. If no explicit constraints are used, this may quickly lead to disorientation. The fact that direction of movement provides the user with another degree of freedom (he or she can look around while moving). This may be confusing for novice users though [Mine ‘95].
[pic]
Figure 3.1 Hand and gaze directed navigation techniques [Mine '95]
2 Crosshairs
The crosshairs technique aims to take away this last problem of the pointing technique. With this technique, the direction of movement is specified by the vector from the user’s head to his or her hand (see Figure 3.1). The user can simply position the cursor (usually the user’s finger or a hand-held device) so that it visually lies on top of the desired destination. A problem with this technique is that it may lead to arm fatigue [Mine ‘95].
3 Two-handed flying
Another hand directed technique is two-handed flying [Mine ‘97a]. Here, the direction of movement is specified by the vector between the user’s hands. Speed is proportional to the distance between the user’s hands (see Figure 3.2). If this distance is smaller than a certain threshold, the user stops moving. To fly backwards, the user can simply swap his hands. Because this technique exploits the user’s sense of proprioception, he or she can easily judge his direction and speed. However, it has also been found to be very difficult to keep a constant speed of motion while changing direction [Mine ‘96].
[pic]
Figure 3.2 Two-handed flying [Mine '97a]
3 Gaze directed techniques
Gaze directed techniques use the head rather than the hand to specify the direction of motion. Three examples of this type are:
• Gaze directed flying
• Orbital mode
• Head behavior based movement
1 Gaze directed flying
In gaze directed flying, the user simply moves in the direction he is looking (see Figure 3.1). This is usually approximated by the position of the head, but eye tracking can be used for more accurate results. A drawback of this technique is that the user can’t look around while he or she is moving. The ability to look around while moving may help the user maintain situation awareness (see paragraph 3.1.2) when changing position in the environment [Mine ‘95].
2 Orbital mode
Orbital mode [Koller ‘96][Mine ‘95] is a method that allows the user to ‘orbit’ a particular center of rotation (usually an object). The distance between the user and the center of rotation determines the radius of the orbit. The viewpoint is constrained to be always directed towards this center of rotation. This allows the user to quickly inspect a particular object from different angles.
[pic]
Figure 3.3 Orbital mode [Mine '95]
By looking up the bottom of the object will be visible; by looking down the top of the object will be visible (see Figure 3.3). If the radius is large, the user can easily cover large distances in the scene.
Although the method uses head movement for specification of direction, the way it does this is not natural. Because of this, it has been found to evoke stronger symptoms of simulator sickness. Another disadvantage is that in scenes with a lot of objects, the center of rotation may often be occluded. However, for quick inspection of a solitary object, the method has been found to be very useful.
3 Head behavior based movement
In [Kheddar ‘95] a technique is described that doesn’t exactly use the gaze direction, but it does use the head to specify direction. The space around the user’s head is divided into a number of regions, each representing a certain direction (forward, back, up, down, left, right) as shown in Figure 3.4. If the user moves his head into one of these regions this will result in movement in the corresponding direction. The area immediately surrounding the user’s head represents a ‘dead’ area. If the user’s head is in this region, he or she will not move. This technique is very intuitive (the user nods into the direction he wishes to go), and it requires no special tools (except for a tracker on the user’s head).
[pic]
Figure 3.4 Division of the space around the user's head [Kheddar '95]
4 Object driven techniques
With object driven techniques, the user doesn’t control his direction directly. Instead, it depends on a particular object in the environment. Examples of this are autonomous vehicles, like a virtual elevator, which transport the user automatically to another position. Attractors and repellors are other examples. An attractor is an object that pulls the user towards ifself. A repellor pushes the user away from itself. These objects may also be placed under user control, so that the user can position them at a desired location, and then let himself be drawn towards that location [Mine ‘95].
5 Goal driven techniques
Goal driven techniques allow the user to specify a desired destination directly, after which he or she is automatically moved there. There are several possible ways for the user to select his destination. The user may be presented with a virtual menu (see Chapter 6) that lists a limited set of possible destinations. Because the user can only move to these destinations, this method is highly constrained [Mine ‘95]. In [Bowman ‘96] an example of this approach, called the ‘flying chair metaphor’ is described. Here, the user sits on a chair and is able to select from seven interesting locations in the environment. When one of these is chosen, the user is automatically flown there through the use of a smooth animation.
Another possibility is to provide the user with a virtual map. This map may again contain a limited set of possible destinations (in the form of icons). Another possibility is that the user can select any point on the map. A special use of a (three-dimensional) virtual map is the World In Miniature (WIM) metaphor, which will be discussed in paragraph 3.2.7.
6 Dynamic scaling
This technique makes use of scaling to move through an environment. Here, the user can either scale the virtual world (or himself) up or down. Navigation can be accomplished in the following way:
First, the user scales the world down (or himself up) until the desired destination is within reach. Here, the user’s position is used as the center of scaling. After that, he moves the center of scaling to this destination. Finally, he scales the world (or himself) back to its original size. As a result of this scaling, the user will now be at the destination location. The advantage of this technique is that the scaled-down world forms a virtual map on which the user can easily locate his destination.
7 World in miniature (WIM)
One of the few interaction metaphors in virtual environments that is fairly well developed is the World In Miniature or WIM metaphor [Stoakley ‘95][Pausch ‘95]. It has been proven to be very successful not only for navigation, but also for the other basic interactions like selection and manipulation (see chapters 4 and 5).
This technique provides the user with a hand-held, three-dimensional miniature of the full-scale virtual environment. This miniature gives the user another point of view onto the virtual world. Because of this ‘bird’s-eye’ view, it is very easy for the user to form a mental model of the structure of the virtual world, thereby increasing his situation awareness. When the user manipulates the small objects in this miniature, the corresponding life-size objects will follow these manipulations accordingly. A picture of a WIM is shown in Figure 3.5.
[pic]
Figure 3.5 World in Miniature (WIM) [WebWIM]
One way to use the WIM for navigation is to place a representation of the user as an object in the WIM. The user can then pick ‘himself’ up and move himself to a new position in the scene. This introduces the problem of when to move the user to the new position in the actual virtual environment. When the user moves his WIM representation, the viewpoint in the life-size virtual environment could be updated immediately. However, this has been found to be highly disorienting. In this case, it is better to first let the user finish manipulating his WIM representation. Only then the actual viewpoint is smoothly animated through the scene to the new location. This allows the user to maintain a better situation awareness. However, automatically determining the animation path may be difficult if the user is not to fly through objects. Another problem is that the distance to be covered may be large. Furthermore, this method was apparently still disorienting to most users because they needed to shift their focus between the WIM and the life-size VE [Pausch ‘95].
An alternative is presented in [Pausch ‘95]. This method is based on the observation that users seemed to identify with their WIM representation. Once the user releases his WIM representation, he is animated into the WIM. In a sense, the user actually becomes his WIM representation, and the WIM becomes the actual virtual environment. The old full scale VE is removed, and the user is presented with a new WIM.
Another interesting use of World In Miniatures for navigation is described in [Pausch ‘95]. Here the user has access to multiple WIMs of the scene. This is useful when the full scale environment is so large that it would be unrecognizable if it would be presented as one WIM. Other possibilities are the need for the user to switch between different views of the same scene (each view displaying different properties of the world), or between different worlds altogether. When the user moves his WIM representation into an other WIM, he is quickly ‘teleported’ to the new context.
3 Specifying speed
Besides direction itself, the speed of motion in that direction must also be specified in one way or the other. Of course, physical and virtual controls can be used for this. Other techniques can be categorized as follows [Mine ‘95]:
• Constant speed
• Constant acceleration
• Hand controlled
1 Constant speed
This very simple technique always uses a constant speed to move the user once he has selected a direction using one of the methods described above. When this technique is used, it is important that the user is able to traverse the entire environment in a reasonable amount of time. Usually this technique is too limited, because it is very hard to successfully navigate to a particular point (due to overshoot).
2 Constant acceleration
When constant acceleration is used the user begins at a slow speed. As long as he keeps moving, he keeps accelerating. This allows the user to quickly reach great speeds, and therefore to fly great distances. Once he reaches his destination he can stop and then restart motion and move slowly and accurately to the exact location. It is important that the rate of acceleration is chosen well.
3 Hand controlled
Another possible technique involves the user’s hand position. The distance of the user’s hand from his head may for example determine speed. The mapping function of distance to speed is typically linear or exponential. Another possibility is to divide the total reach of the user’s arm into three regions. When the user’s hand is in the first region, he decelerates. When in the second region, the current speed is maintained, and when in the third region, the user accelerates. A drawback of this method is that it may lead to arm fatigue. These methods are illustrated in Figure 3.6.
[pic]
Figure 3.6 Using the hand to specify speed [Mine '95]
2 Predictor based technique
An interesting technique that aids the use of a physical control to specify speed and direction of motion is described in [Chapman ‘92]. Here, a ‘predictor window’ is used that predicts where the user will be in a short time, based on the current speed and direction of motion. This predictor is shown to the user as a rectangular window. To arrive at a particular destination the user only has to point the predictor at it. It has been found that this technique is especially helpful to novice users (who tend to move slowly). This is because more experienced users move so fast and in such sharp angles that the predictor is projected outside of the user’s field of view. Therefore the user only sees glimpses of the predictor, which has been found to be rather distracting. A solution might be to predict the user’s position a shorter time in the future.
4 Navigation in CHIMP and the Virtual Gorilla Exhibit
In this paragraph the methods that have been chosen in the two applications introduced in chapter 1, CHIMP and the Virtual Gorilla Exhibit, will be discussed.
1 CHIMP
In CHIMP the user is provided with two methods of navigation: pointing-mode flying and orbital mode. As was discussed in paragraph 3.2.3, orbital mode allows the user quick inspection of an object
[pic]
Figure 3.7 WIM used in CHIMP [WebCHIMP]
from all sides by simply moving his head. This is very useful in architectural design. Before deciding on whether (and where) to place a particular object in the scene, the user can quickly inspect the object to see if it meets his demands. Once the object is positioned in the environment, the user can orbit around it and assess if it ‘fits’ into the scene from an architectural point of view.
The pointing-mode flying technique is useful in almost any application because it allows the user to fly towards a particular object or point in the scene by using a very intuitive method, i.e. pointing at the destination. This allows the user to easily look at the scene from different points of view.
CHIMP also provides the user with a World In Miniature (see paragraph 3.2.7). This global point of view can help the user evaluate his overall design. It is interesting to note that CHIMP combines the WIM metaphor with orbital viewing to provide an orbital WIM. Here the WIM is normally left floating in space. If the user wishes to view the WIM from a different direction he can activate the orbital mode. One advantage of this is that it maintains correspondence between the orientation of the WIM and of the virtual world. This avoids the need to mentally rotate the WIM to determine which direction in the WIM corresponds to ‘north’ in the full scale world (like when reading a map when driving in a car). A picture of the WIM used in CHIMP is shown in Figure 3.7 [Mine ‘96].
2 Virtual Gorilla Exhibit
Navigation in the original Virtual Gorilla Exhibit is accomplished by ‘virtual walking’. The user holds a joystick, and by using the joystick’s buttons the user moves forward or backward based on gaze direction. A heightfield is used to maintain a constant height above the terrain [Allison ‘96]. Simple collision detection is also provided, so that the user can’t walk through rocks and trees.
[pic]
Figure 3.8 Virtual tablet used in the Virtual Gorilla Exhibit [WebGORILLADESIGN]
In the extended Virtual Gorilla Exhibit (for habitat design), the user is provided with a virtual tablet on which a (two-dimensional) map of the environment is shown. On this map, the user’s location is represented by an icon. The joystick is replaced by a stylus, and the user can ‘drag’ his iconic representation across the map using the stylus. Figure 3.8 shows a picture of the tablet. The red dot represents the user. The figure also shows the virtual representation of the stylus with which the user can drag the red dot to a new location on the map. The gaze directed technique from the original Exhibit has been replaced by a pointing technique using the stylus so that the user can look around while navigating.
The user is no longer constrained to move at a constant height above the terrain, but can fly upward to get a bird’s eye view of the habitat. These changes were made because they aid the user in evaluating the design of the habitat.
4 Selection
This chapter will discuss the second basic interaction task, which is the task of selection. It has roughly the same structure as the previous chapter. In paragraph 4.1 some issues concerning the selection of virtual objects will be discussed. This is followed by a description of several selection mechanisms that have been found to be useful in virtual environments (paragraph 4.2). The chapter will be concluded with an overview of the selection techniques implemented in CHIMP and the Virtual Gorilla Exhibit (paragraph 4.3).
1 Issues of selection
In general, selection is the task of identifying an object. It is also referred to as target acquisition. This task involves two separate actions [Mine ‘95]:
• Identification of the object that is to be selected.
• Issuing of a signal to indicate the actual selection.
In the next paragraph several methods to perform the first action will be discussed. The second action may be performed for example by pushing a button, giving a voice command, or making a gesture.
A user may want to select an object to peform a command on it, to group related objects, or to manipulate the object [Bowman ‘95]. Although the tasks of selection and manipulation are closely related, it is important that they be considered separately. It has been found that methods that make selection easy are not necessarily easy for object manipulation as well [Bowman ‘97].
Some parameters that influence the selection task are [Poupyrev ‘97]:
• Number of objects that is to be selected
• Distance of the user to the target object
• Size of the target object
• Direction to the target object
• Occlusion of the target object
• Other parameters, like the dynamics of the target, the number of surrounding objects, and the shape of (the bounding volume of the) target object
An example of how these parameters affect selection is the fact that a small, distant object is usually much harder to select than a large, nearby object.
An important problem that a good selection technique should deal with is the limited range of the user (in other words, the user should be able to select objects that are out of his reach just as easily as objects that are within reach). Another problem is the limited precision of the user’s hand, which makes it difficult to select very small, or highly cluttered objects. This last problem is made worse by limitations in technology, like tracker noise and low display resolution. A possible solution for this would be to allow the user to work at different scales (e.g. zooming in and out on objects) [Mine ‘95]. Finally, occlusion of the target is a big problem. Often the target object will be partially or completely obscured by other objects, which makes it harder to select it. It also increases the odds of selecting the wrong object. Therefore it is usually necessary for the user to be able to easily change his view point in the scene to visually isolate the target object [Poupyrev ‘97].
In any case, each method should provide the user with good feedback. The user must know when he has chosen an object for selection. This can be done by highlighting the object or showing its bounding box. Furthermore, he must know when he has successfully performed a selection action, for example by providing him with audible and visual cues. Finally, the user must at all times be able to determine the current selection state of all objects (e.g. by rendering selected objects with a distinct color) [Mine ‘95].
2 Selection techniques
In this paragraph, several techniques will be described that have been used in virtual environment applications. The following categories will be distinguished:
• Physical intersection
• Scaled-world grab
• Silk cursor
• Ray casting techniques
• Arm extension techniques
• Image plane techniques
• Other selection techniques
1 Physical intersection
Like physical movement was for navigation, physical intersection is the most natural method for selecting objects. Here, an object is chosen for selection by placing a cursor inside it. This cursor is usually attached to the user’ hand [Mine ‘95]. After this, a predefined signal can be used to perform the actual selection, as was described in paragraph 4.1. An illustration is shown in Figure 4.1.
Although this technique is intuitive, an obvious drawback is of course that only objects that are within arm’s reach can be selected. To reach distant objects the user is required to navigate towards them, which can be very time consuming. This forms a good example of the fact that increased naturalness may not always lead to better performance. Another problem with physical interaction is that the human hand is not precise enough when very small or densely cluttered objects have to be selected [Bowman ‘95][Pierce ‘97].
[pic]
Figure 4.1 Physical intersection [Mine '95]
2 Scaled world grab
To accommodate for the limited range of physical intersection, scaled world grab can be used. With this technique, the complete environment is scaled down about the user’s head once he reaches out for an object. Because the scene is scaled about the user’s head he won’t notice that scaling has taken place, unless he moves his head (after scaling he will be able to see both sides of a big object like a house by small head movements left-to-right). If the world is scaled down far enough, the user will be able to grab even the remotest objects. When the user releases the object, the environment is scaled up again [Mine ‘97a]. The technique is illustrated in Figure 4.2.
[pic]
[pic]
Figure 4.2 Scaled world grab [Mine '97a]
3 Silk cursor
The silk cursor technique was developed for dynamic target acquisition in non-immersive environments. It can be seen as the non-immersive version of physical intersection. Basically, it is an extension of the normal mouse pointer to a three-dimensionsal virtual cursor. Instead of a single point, a volume is used as the cursor. Furthermore, this volume is made to be slightly transparent. This provides the user with valuable depth cues that have been shown to increase performance. The silk cursor is shown in Figure 4.3. An object at position A is seen through two layers of the volume, so it appears to be behind the cursor. An object at position B is seen through only one (slightly transparent) layer, and is perceived to be inside the volume (and can thus be selected). Finally, an object a point C is not occluded by the cursor at all, so it is located in front of it [Zhai ‘94].
[pic]
Figure 4.3 Silk cursor technique [Zhai '94]
4 Ray casting techniques
As was argued in paragraph 4.1, the user should have some in some way be able to select objects that are out of his physical reach. Ray casting techniques (see Figure 4.4) solve this problem by using a virtual ray that extends from the user’s hand into the environment. The first object that intersects the ray can then be selected. As long as the user can point at an object it can be selected, because the intersection is a single point and the ray can be infinitely long [Bowman ‘95]. However, because of the limited precision of the user’s hand and possible other causes like tracker noise, selection of distant or small objects can be difficult. This is caused by the fact that small hand movements result in large displacements of the tip of the ray.
[pic]
Figure 4.4 Ray casting [WebGRABBING]
1 Cone casting
To deal with this last problem (lack of precision) when selecting distant or small objects, a cone or ‘spotlight’ is sometimes used instead of a ray. Since the width of this cone expands with distance from the user, far-away objects are just as easy to select as nearby ones. However, because of its finite width, some method has to be provided to allow the user to choose when more than one object falls within the volume of the cone [Bowman ‘95][Pierce ‘97].
Once an object is selected using these techniques, it is usually attached to the ray. However, it is very awkward to manipulate an object this way. In [Bowman ‘97] an extension called the ‘fishing reel’ metaphor is described where the user can push buttons to ‘reel’ objects closer or farther away.
A problem that remains for both ray and cone casting is that of occlusion. To be able to select an object, it has to be in the direct ‘line-of-sight’ of the ray. Therefore, these techniques usually should be coupled with a method that allows for quick changes in the user’s position (e.g. orbital mode which was described in chapter 3). This somewhat negates the advantage of the user not having to navigate in order to select (distant) objects [Poupyrev ‘96][Stoakley ‘95].
5 Arm extension techniques
Arm extension techniques take a slightly different approach towards solving the problem of selecting distant objects than the ray casting techniques described above. Instead of letting the user manipulate a virtual ray that is attached to his arm, the user can make his virtual arm grow to an arbitrary length. This allows him to select and manipulate an object with his hand, which is a very natural mapping [Bowman ‘97]. To be effective, an arm-extension must provide an anwer to the following questions:
• How does the user specify when he wants to extend his virtual arm
• How does the user control the length of his virtual arm
Besides this the method should, like all interaction techniques, be intuitive, seamless, and easy to use.
The most important arm extension technique is the so-called Go-Go technique. This technique, as well as several variations on it, will be described in the next sections.
1 Go-Go technique
The Go-Go technique allows the user to extend his virtual arm by using a non-linear function that maps the real hand position to the virtual hand position. As can be seen in Figure 4.5, the space around the user is divided in two parts. The horizontal axis represents the user’s real hand position; the vertical axis represents the virtual hand position. When the user’s real hand is close to his body the mapping function is linear (i.e. the virtual hand is at the same position as the real one). However, when the user moves his real hand farther away from his body, the mapping function becomes non-linear and the virtual hand moves faster outwards than his real hand. Because of the continuity between these two functions, the transition is very smooth. Figure 4.6 shows the Go-Go technique in action. The small white cube represents the position of the user’s real hand [Poupyrev ‘96].
[pic]
Figure 4.5 Mapping function used in the Go-Go technique [WebGOGO]
The fact that the Go-Go technique imitates real world behavior (to reach out towards an object we want) makes it very intuitive. It is also very easy to use in that the user only has to extend his real arm to extend his virtual arm. Furthermore, the technique allows the user to use the same technique for selection of both close and distant objects [Poupyrev ‘96].
A disadvantage of this technique (as well as all the other arm extension techniques) is that the virtual hand actually has to be inside of an object to be able to select it. This can be a problem when selecting small, distant objects [Bowman ‘97].
2 Stretch go-go
Another drawback of the original Go-Go technique described above is that it still has a finite range, determined by the length of the user’s arm. For each environment that has a different size (and maybe even for every user), the mapping function would have to be adjusted in order for the user to be able to reach objects that are at the other side of the environment. To accommodate for this, the stretch go-go
[pic]
Figure 4.6 Go-Go in action [WebGOGO2]
[pic]
Figure 4.7 Stretch go-go [Bowman '97]
technique has been developed. Here, the space around the user is divided into three regions. When in the middle region (which is the one where the user’s hand is normally in), the virtual arm length stays the same. When the user’s real hand is in the outermost region, his virtual arm will grow at at constant speed. Finally, when the user’s real hand is in the innermost region, his virtual arm will shrink. The stretch go-go technique is shown in Figure 4.7. At the right of the screen a gauge is shown that tells the user in which of the three regions his arm currently is. This technique is less intuitive than the normal Go-Go technique, but the virtual arm is able to grow indefinitely [Bowman ‘97].
3 Other variations
Several other variations of the Go-Go technique are possible. One possibility is to use buttons that allow the user to make his virtual arm grow or shrink at a constant speed. This technique, called indirect stretching, is again less intuitive but may be more precise. Other possibilities include:
• Mapping physical hand velocity to the rate of growth of the virtual arm.
• Adding buttons to this control velocity.
• Generalization of stretch go-go to allow a continuous range of virtual arm velocities.
Although they are more powerful, these techniques are more complicated to use than the original Go-Go and the stretch go-go techniques [Bowman ‘97].
6 Image plane techniques
An altogether different approach towards object selection is described in [Pierce ‘97]. Several methods will be discussed that all use the two-dimensional projections that (distant) three-dimensional objects make on the image plane. They are:
• Head crusher technique
• Sticky finger technique
• Lifting palm technique
• Framing hands technique
Basically, what these techniques do is to find the position of the user’s hand on his image plane and then to use this position to cast a (invisible) pick ray into the scene. The methods differ in how the position of the hand is determined.
1 Head crusher technique
In this technique, the user positions his thumb and forefinger so that it seems that the object is ‘pinched’ between these fingers, as shown in Figure 4.8. A pick ray is then cast from the user’s eye point through the point midway between the user’s fingers to determine which object has been selected.
[pic]
Figure 4.8 Head crusher [WebCRUSHER]
[pic]
Figure 4.9 Sticky finger [WebCRUSHER]
2 Sticky finger technique
The head crusher technique is not very useful when the user tries to select very large objects or objects that are very close. Here the sticky finger technique is more appropriate. The user picks an object by
simply placing his index finger over it. A ray is again cast from the user’s eye point through the top of the user’s index finger to determine the selected object (see Figure 4.9).
3 Lifting palm technique
Yet another image plane selection technique is the lifting palm technique. Here the user can select an object by flattening his hand and positioning his palm below the object so that it appears to stand on his hand. A ray is cast from the eye point through a point slightly above the user’s palm to determine which object has been selected. This technique is illustrated in Figure 4.10.
[pic]
Figure 4.10 Lifting palm [WebCRUSHER]
[pic]
Figure 4.11 Framing hands [WebCRUSHER]
4 Framing hands technique
Finally, the framing hands technique uses both of the user’s hands. The user places his hands in such a way that they form a frame that surrounds the desired object, as shown in Figure 4.11. The user can arbitrarily rotate and resize the frame at any time. The frame can also be made visible to provide feedback to the user. Again, a ray is cast from the user’s eye point through the midpoint between the user’ s hands to find the selected object.
When a few possible candidate objects result after one of the described techniques has been used, the object whose orientation best matches the orientation of the user’s hands could be chosen. In the head crusher, the sticky finger, and the framing hands techniques, this orientation is provided by the fingers; the lifting palm can use the normal to the palm of the hand.
A problem of the image plane techniques are the image plane of which eye to use when the scene is rendered in stereo. In this case it is possible that a different object lies below the user’s hand in each plane. Other problems are arm fatigue and the fact that the user’s hands may obscure small objects. The fact that the user has to switch between the notion of a ‘real’ three-dimensional environment and a two-dimensional view doesn’t seem to be a problem [Pierce ‘97].
7 Other selection techniques
In this paragraph some other techniques that have been used for object selection will be briefly discussed. These techniques are:
• Gaze directed selection
• WIM
• Indirect selection
1 Gaze directed selection
In gaze directed selection, the user simply looks at an object, after which he gives on of the standard signals to perform the actual selection (like a voice command, pushing a button).
2 WIM
Besides being useful for navigation, the World In Miniature can of coure be used for selection as well. The user can easily reach into the WIM and select any object. Because of the panoramic view the WIM provides, all objects will in general be visible. If an object is obscured, the user can easily rotate the WIM to make it visible. A problem is of course the miniature size of the objects in the WIM, which may make it hard to select them [Stoakley ‘95][Bowman ‘97].
3 Indirect selection
When indirect selection techniques are used, the user doesn’t select the object itself. Instead, he for example gives a voice command plus the name of the object (e.g. ‘select chair’). Another example is the use of a menu with the names of each object from which the user can select one. The problem with these techniques is that the user has to remember the names of all the objects to be able to select them. The user doesn’t have to see the object in order to select it though [Mine ‘95].
3 Selection in CHIMP and the Virtual Gorilla Exhibit
Now the selection methods used in CHIMP and the Virtual Gorilla Exhibit will be examined.
1 CHIMP
Besides physical intersection, where the user can simply reach out and grab objects that are within arm’s reach (see paragraph 4.2.1), CHIMP provides the user with a spotlight to select objects that are farther away, as was discussed in paragraph 4.2.4. When the spotlight intersects the desired object, a trigger button is pressed to perform the actual selection. Inspired by the silk cursor technique (paragraph 4.2.3), the spotlight in CHIMP is made partially transparent to provide extra depth cues.
[pic]
Figure 4.12 Spotlight used in CHIMP [WebCHIMP]
Furthermore, the spotlight changes color when an object can be selected, thus providing the user with valuable feedback. The use of the spotlight in CHIMP is illustrated in Figure 4.12.
As a third technique, the WIM provided to the user by CHIMP can be used for selection as well, for example when an object is occluded from the current point of view in the virtual environment.
2 Virtual Gorilla Exhibit
Two selection methods are provided in the (extended) Virtual Gorilla Exhibit. One is the Go-Go technique where the user’s virtual arm stretches at non-linear rate compared to his physical arm (see paragraph 4.2.5). This enables the user to easily select trees and rocks that are far away. When the virtual hand is inside an object (and that object can be selected) this is indicated to the user by highlighting of the object. The second selection technique uses the virtual tablet that was introduced in chapter 3. The user simply places the stylus on the iconic representation of an object and presses the button of the stylus to select the object. The corresponding object in the actual environment is highlighted as well when this indirect technique is used [WebGORILLADESIGN].
5 Manipulation
In this chapter the third and final basic interaction task will be discussed, which is the manipulation of virtual objects. In paragraph 5.1, various issues and problems concerning virtual object manipulation will be described. After that follows a discussion of several techniques for manipulating objects in a VE in paragraph 5.2. In paragraph 5.3, the manipulation techniques used in CHIMP and the Virtual Gorilla Exhibit will be investigated.
1 Issues of manipulation
This paragraph will discuss some of the issues and problems concerning manipulation of virtual objects in VEs. First of all, manipulation will be defined, and some examples of different manipulation tasks will be given in paragraph 5.1.1. After that, the importance of using constraints in manipulation will be explored in paragraph 5.1.2. Finally, the characteristics and advantages of two-handed manipulation will be discussed in paragraph 5.1.3.
1 What is manipulation?
Manipulation is the task of changing one or more parameters of a particular object. The number of ways in which objects can be manipulated is almost unlimited. The most important manipulation task is probably the positioning of objects; other examples include changing an object’s size, color, or shape, creating and deleting objects, or grouping objects [Bowman ‘95]. As always, the manipulation tasks that have to be available to the user in a particular VE application depend entirely on the task he has to perform in the VE. This chapter will focus on the task of changing the position and orientation of an object, since this will be necessary in almost every VE application.
The positioning of an object in three dimensions requires the user to specify six parameters: three for the location of the object, and three for its orientation. This task can be subdivided into three phases [Poupyrev ‘97]:
• Selecting the object
• Primary or gross positioning/orienting
• Fine adjustment to achieve the desired accuracy
For the selection of an object, one of the techniques described in chapter 4 can be used. After that, the user first moves the object grossly to the desired location. Finally, he fine-tunes the position and orientation of the object depending on the accuracy that is required. Especially this last phase is very hard in virtual environments. This has the following causes [Mine ‘97a]:
• Limited input information
• Lack of constraints
Besides the obvious problem of technological limitations like tracker noise, another problem is the limited input information that is generally available. Usually only the user’s head and hands are tracked, and a few buttons or a glove are available. In the real world on the other hand, people use complex tools to manipulate objects, as well as all of the individual fingers, and not just the hand as a whole. The other problem, the lack of constraints, is the most imporant issue, and will be discussed next.
2 Constraints
As was already mentioned in chapter 2, one of the biggest problems that makes precise manipulation in virtual environments hard is the general lack of constraints. Examples of real-world constraints that restrict a user’s movements are gravity, work surfaces like table-tops, and the surfaces of the objects he wishes to manipulate. People are used to these constraints and rely on them to perform tasks in the real world [Stoakley ‘95]. The problem of the lack of constraints in VEs has several different aspects:
• Lack of haptic feedback
• Inaccurate collision detection
• Multiple degrees-of-freedom
1 Lack of haptic feedback
The most obvious aspect is the lack of haptic feedback in virtual environments. Although a lot of research is being done in the field of haptics, very few VE systems actually contain such devices. The user is unable to feel or touch the surface of a virtual object, simply because nothing is there. Therefore, and because humans lack the fine motor control, it is very difficult to precisely manipulate virtual objects. Furthermore, it is impossible to store knowledge on how to manipulate objects in the environment itself. In the real world, the physical constraints of objects give clues on how to use the object. For example, a dial can only be rotated about its axis. Virtual objects don’t have these inherent constraints, so they have to be explicitly programmed into the application [Mine ‘97a].
2 Inaccurate collision detection
Another problem is collision detection. General collision detection is often the most desirable. This means that no object can pass through another object. However, this requires a lot of computation, which is not always feasible. Therefore, most algorithms used are based on collision checks between bounding volumes of objects. This is relatively fast, but not very accurate and leads to very unnatural interaction [Bowman ‘95][Figueiredo ‘93].
3 Multiple degrees-of-freedom
Another aspect of the lack of constraints is the so-called degrees-of-freedom problem. In complex virtual environments, the number of parameters that can be placed under user control is virtually unlimited. If the user can change these parameters all at the same time, this will probably have a negative effect on productivity. Often, not enough attention is paid to the organization of the input operations to make sure that the number of parameters that can be affected at any one time is severely limited [Barfield ‘95].
As was stated above, the positioning of objects requires six degrees-of-freedom to be specified. If a task requires any level of precision, the number of degrees-of-freedom that can be changed at any one time has to be restricted. Redundant methods for manipulating objects are useful here, where each method poses additional constraints on the manipulation. The user can then use an unconstrained method for the fast initial positioning of the object, and subsequently apply more constrained methods to achieve the desired accuracy. The challenge here is to provide an interface which effectively integrates these redundant techniques, and not confuse the user. Restricting the number of constraints is not only important for positioning tasks, but also for more abstract tasks, like changing the color of an object. These tasks should be constrained to one- or two-dimensional tasks. [Bowman ‘95][Hinckley ‘94].
3 Two-handed manipulation
Although some tasks are unimanual (i.e. only use one hand, like brushing one’s teeth), a lot of real world tasks are performed using both hands. Two categories of two-handed tasks can be distinguished [Cutler ‘97]:
• Bimanual symmetric
• Bimanual asymmetric
An example of a bimanual symmetric task would be weightlifting. Most common tasks fall in the category of bimanual asymmetric tasks, an example of which is dealing cards. The orthogonal division of labor, where both hands perform unrelated, unimanual tasks could be considered as a category as well. Studies have shown that such behavior, where users perform two unrelated tasks in a synergistic fashion, often occurs naturally [Cutler ‘97].
Traditionally, two-handed interaction has been seen as a way to perform two subtasks in parallel. However, if the two-handed interaction is well-designed, the use of both hands to perform one task can reduce the number of actions performed, reduce cognitive load, and increase performance. Furthermore, two-handed techniques seem to be more easily learnt. This is because in everyday life meaning is often conveyed by using two-handed gestures, for example by moving the hands together or apart to indicate size [Cutler ‘97][WebHINCKLEY].
The roles of the preferred and the non-preferred hand in bimanual asymmetric tasks have been found to be quite different [Cutler ‘97][Stoakley ‘95]:
• The non-preferred hand generally provides a spatial frame of reference, and the preferred hand performs its actions relative to this reference frame.
• The preferred hand is capable of performing very precise gestures, while the non-preferred is specialized in larger movements.
• The non-preferred hand generally initiates the action.
People require far less concentration to manipulate objects relative to each other than when one object is fixed in space. By holding an object in the non-preferred hand, a reference frame is provided for the preferred hand to easily manipulate the object. When designing two-handed interaction techniques these three characteristics should be taken into account.
2 Manipulation techniques
This paragraph describes two general methods that allow manipulation of virtual objects. In paragraph 5.2.3, some techniques that have been developed to deal with the general lack of constraints in VEs will be discussed.
Once an object is selected, the user can manipulate it by using one of the following two general methods [Rygol ‘95]:
• Direct manipulation
• Tools/widgets
1 Direct manipulation
People are used to manipulating objects with their hands all their life. Therefore, the most desirable way to manipulate virtual objects would seem to be able to grab and move the objects with the hands using very natural gestures and motions, just like in the real world. This is called direct manipulation [Mine ‘95]. This method of manipulating objects gives the user a very direct and precise sense of control, and allows him to exploit his proprioceptive sense. Although direct manipulation techniques are very useful for coarse manipulation of virtual objects, these methods suffer greatly from the lack of constraints, and in particular the lack of haptic feedback. Therefore, manipulation is usually based on hand gestures (e.g. rotating the hand rotates the object). A better approach would be to base manipulation on physics (individual fingers apply forces to the object, allowing the user to ‘roll’ the object through his fingers), but without haptic feedback this is impossible [Kijima ‘96].
[pic]
Figure 5.1 Hand-centered direct manipulation [WebVRMAT]
Just like with the selection of objects, the user should be able to easily manipulate distant objects. After a distant object is selected, object manipulation can either be hand-centered or object-centered. These methods differ in the center of rotation (or scaling) that is used. Another possibility would be to allow the user to specify a center, for example by placing an icon at the desired center [Mine ‘95]. An example of a hand-centered technique for the manipulation of distant objects is the use of a ray casting method to select an object, after which the object is attached to the ray. By moving and rotating his hand (and consequently the ray), the user can manipulate the object. However, it is very difficult to effectively manipulate an object that is attached to a ‘stick’ because of the ‘arm lever’ problem, where a small movement of the hand results in a large displacement of the tip of the ray. Therefore, arm extension techniques are better, because with these the object is placed in the user’s (virtual) hand, which allows for the direct mapping of hand movements to object movements [Mine ‘97b][Bowman ‘97]. In Figure 5.1 a picture is shown where the Go-Go technique is used to allow direct manipulation of virtual objects.
2 Tools and widgets
Besides holding the object in his hands to manipulate it, the user can also use tools to perform this task.
Tools are those objects in the VE that assist the user in performing work within the virtual environment. According to [Bowman ‘95], tools should be:
• Limited in their function
• Limited in their number of configurations
• Limited in their degrees of spatial freedom
• General and reusable
It is important that tools are not made too general though, because the lack of constraints can have a negative effect on their usability. Often the term widget is used to denote the (three-dimensional, graphical) interface element that specifies the appearance of a tool and its method of operation. Using this definition, a tool is then a widget associated with behavior. Widgets are very useful for allowing constrained manipulation. They may be presented to the user in several ways [Rygol ‘95]:
• Fixed spatial location
• Default spatial location
• User-centered presentation
When the objects are in a fixed spatial location, they are immovable and can probably be used to manipulate a nearby object. A good example is attaching manipulation handles to an object, where each handle represents a particular, constrained manipulation task (e.g. rotation about a particular axis). A problem might be that the object can get obscured if a lot of widgets are associated with it, as well as the fact that the user has to be near the object to control its widgets or use a (generally hard-to-use) selection technique to control the widgets from a distance.
When tools are at a default spatial location, the user picks them up from this ‘known’ location in the environment when he needs them. A disadvantage may be that the user has to travel through the environment to get to a tool.
[pic]
Figure 5.2 Hand-held widget [Mine '97a]
User-centered tools are attached to the user in some way, like he is wearing a tool belt. Here, the proprioceptive sense of the user may be exploited to hide the tools at a fixed location relative to the user’s body, just out of his field-of-view. When the user needs a tool, he can just grab for it at the known location. One possibility then is to use the tool on a selected object as a hand-held widget (like a remote control). This allows for easy manipulation from a distance (see Figure 5.2) [Mine ‘97a]. Of course, it is also possible to use the tool just like real-world tools, for example to deform a surface by ‘sculpting’ it with the tool as shown in Figure 5.3. This figure shows the use of a virtual spherical tool to indent an object’s surface.
[pic]
Figure 5.3 Object modification with a virtual tool [Dani '96]
3 Implementing constraints
Because of the limited precision and high costs of current haptic devices, other methods are often necessary to deal with the problem of the lack of constraints in virtual environments. This paragraph will discuss two specific techniques that try to achieve this. They are:
• Physical props
• Precise collision detection
1 Physical props
To give the user haptic feedback and to provide natural constraints, the use of physical props can be useful. Examples include physical tabletops or palettes to constrain motion of objects to a plane, but it is also possible to use physical props that represent the various tools that can be used to manipulate objects, or even the objects themselves in direct manipulation tasks [Mine ‘97a][WebHINCKLEY]. Besides providing haptic feedback, physical props also provide kinesthetic feedback due to their inertia and the force of gravity. Furthermore, their mass tends to damp instabilities in the user’s hand motion [Hinckley ‘94]. Physical props should be designed in such a way that they provide the user with information on how to hold and use the tool (affordances). It is also important that the registration
[pic]
Figure 5.4 Physical props [WebHINCKLEY]
between the physical prop and its virtual representation is very accurate [Mapes ‘95]. A problem of physical props may be that they lead to fatigue more quickly, or that they may get in the way of the user when he tries to perform a certain task [Stoakley ‘95]. An example of the use of physical props in neurosurgical planning is shown in Figure 5.4. Here the user holds a doll’s head which represents the patient’s head in his left hand, while he holds a small plastic plate representing a cutting plane tool in his right hand. This allows the user to interactively view arbitrary cross-sections of the patient’s brain using a very intuitive, two-handed technique [WebHinckley].
2 Precise collision detection
As was discussed in paragraph 5.1.2, most collision detection algorithms are based on bounding volumes, which is not very accurate. In [Figueiredo ‘93] an algorithm is described that performs precise collision detection based on the object’s polygonal information rather than bounding volumes. This has the following advantages:
• Exact determination of whether two objects intersect.
• More natural interaction.
The first point allows the user to manipulate two objects more easily (for example inserting one object into the other). Interaction is more natural, because the user can get as close to an object as in the real world. This allows him to realistically push, grab, and release an object. The technique allows the user to ‘surround’ the object’s surface with his virtual hand, looking for the most natural way to hold the object. A drawback is that this method requires more computation.
3 Manipulation in CHIMP and the Virtual Gorilla Exhibit
1 CHIMP
CHIMP provides the user with several methods for manipulation of objects. A lot of these are two-handed techniques. The following will be explored in this paragraph [Mine ‘96]:
• Action at a distance
• Constrained object manipulation
• WIM
1 Action at a distance
As was described in chapter 4, a spotlight can be used to select objects that are out of the physical range of the user. Once an object is selected, manipulation can be either object-centered or hand-centered (see paragraph 5.2.1).
2 Constrained object manipulation
In an earlier version, CHIMP made use of widgets that were attached to the object they were controlling. However, several problems were encountered with this, such as difficulty of selecting the widgets and visual clutter. Therefore, these widgets were replaced by two-handed techniques which allow the user to perform the following constrained object manipulations:
• movement along a line or in a plane
• rotation about any vector
• rotation about the object’s center
• uniform scaling about the object’s center
The user can select one of these methods by using a remote control, which is a hand-held widget that can be used to interact with objects from a distance. The remote control is called the mode bar. It is a thin bar that is attached to the user’s left hand. The bar is divided into colored subregions, where each region corresponds to a particular manipulation. The bar always lies along the vector between the user’s hands. Therefore, the user is required to only specify one parameter (the distance between his hands) to select a particular manipulation. Furthermore, this technique makes use of the principles of two-handed manipulation. The mode bar is shown in Figure 5.5.
[pic]
Figure 5.5 Mode bar for constrained manipulation in CHIMP [WebCHIMP]
Once the user has selected a particular manipulation technique with the mode bar, the user specifies the parameters of the manipulation itself in the following way:
• Specification of the axis of the manipulation.
• Specification of the magnitude of the manipulation.
First, the orientation of the right hand specifies the axis of the interaction, which is confirmed by the press of a button. The user can specify if the specified axis should snap to the closest of the principal axes. After that, the magnitude of the manipulation depends on the relative motion of both hands. For translation for example, the right hand is moved along the specified axis to move the objects. For rotation, the user twists his right hand about its axis. For scaling, the size of an object is determined by the distance between both hands.
3 WIM
Finally, the World In Miniature that is available in CHIMP can be used to manipulate objects as well.
When manipulation is done using a WIM, the environment essentially becomes its own (hand-held) widget [Stoakley ‘95]. Again, the principles of two-handed manipulation are adhered to: the WIM is held in the non-preferred hand to provide a reference frame for the dominant hand which picks up the miniaturized objects and moves them around. A problem with using the WIM is of course the limited precision that is possible and the size of objects when the environment is large [Bowman ‘97].
CHIMP also uses a lot of virtual menus and control panels for interaction. These will be discussed in chapter 6.
2 Virtual Gorilla Exhibit
In the (extended) Virtual Gorilla Exhibit, the only manipulations that are possible are the creation of new rocks and trees in the environment, and the movement of these trees, rocks, and bushes to a new location [WebGORILLADESIGN].
The user can create new objects by using the virtual tablet. He can drag a new copy of an icon of the particular object into the map of the environment with the stylus. Objects can be moved around by selecting their icon on the tablet and moving it around. This is obviously a highly constrained method where a physical stylus is dragged over the physical tablet the user is holding to provide haptic feedback to the user. This is a good example of the use of physical props to enhance manipulation.
Objects can also be moved around by selecting them in the immersive environment (by using the Go-Go technique) and then moving the arm to place the object at the desired location. The objects can not be rotated, and the application always makes sure that objects remain on top of the terrain’s surface. This is an example of the use of software constraints to ignore extraneous degrees-of-freedom of the user’s input: of the six degrees of freedom the user can input with his hand, only two (translational) degrees-of-freedom are being used. Finally, (crude) collision detection is used to ensure that trees and rocks can not intersect with each other.
6 Virtual menus
This chapter will discuss the use of menus in virtual environments (called virtual menus). First of all, some problems and issues concerning the use of menus in these environments will be discussed in paragraph 6.1. After that, a look will be taken at possible implementations of virtual menus that try to solve some of the problems associated with them in paragraph 6.2. In paragraph 6.3 the ways in which virtual menus have been used in CHIMP and the Virtual Gorilla Exhibit will be described.
1 Issues of virtual menus
The previous three chapters discussed the basic interactions navigation, selection, and manipulation. The methods described to perform these tasks were almost all direct interaction techniques, in that the user could specify the parameters of the desired task directly through the use of natural gestures such as pointing or grabbing. In this chapter a different, indirect approach will be discussed that makes use of menus to allow the user to interact with and give commands to the environment.
This paragraph starts off with a definition of a (virtual) menu and a discussion of some of the concepts of conventional menus used in traditional desktop computing environments. After that, the ways in which these concepts translate into virtual environments will be examined, together with some of the problems this may introduce.
1 What are (virtual) menus?
A computer menu is a collection of items, usually options and commands, that is displayed to the user and from which he can interactively select an item. Typically, a menu takes the form of a sequential list of text items that is arranged in a rectangular format However, instead of text icons or other graphics may be used as well. Other arrangements, such as circular, have also been used [Jacoby ‘92].
Two of the most widely used types of menus are [Darken ‘94][Jacoby ‘92]:
• Pull-down menus
• Pop-up menus
1 Pull-down menus
A pull-down menu can be ‘pulled down’ from a menu bar by the user. On the menu bar, the names of all the available menus are displayed. This relieves the user of the burden of having to remember all available menus. After the user has selected a command or option from the menu, it usually disappears again, but this isn’t necessarily the case. The menu bar generally has a fixed position, so the pull-down menus always appear at the same location.
2 Pop-up menus
A pop-up can be made to ‘pop up’ at the current location of the cursor. Because no menu bar is used, the user must have some way to specify which menu should be displayed. An option is to pop up a main menu from which the other menus can be selected.
The main aspects of menus that should be considered when designing a menu system are:
• Invocation
• Location
• Reference frame
• Cursor
• Highlighting
• Selection
• Removal
Invocation concerns the actions the user has to take to display the menu and make it ready for use. Location determines where the menu appears. Another thing to determine is the reference frame of the menu: once visible, is the menu fixed in space or does it move with the user? Other aspects are whether a cursor is used to help the user with pointing, how the menu items are highlighted, and how a menu item can be selected once it is highlighted. Finally, the menu can be made to disappear once a selection has been made, or it can remain visible [Jacoby ‘92].
Menus have been proven to be extremely useful in traditional (desktop) computing environments, and therefore it makes sense to consider using virtual menus, their analogy in virtual environments.
2 Advantages of virtual menus
For certain tasks, such as navigation, selection, and manipulation, the use of direct manipulation techniques in virtual environments seems very obvious. This is because these tasks have real world counterparts, and the interaction techniques used in the VE try to mimic the way in which these tasks are performed in everyday life. However, sometimes the user has to perform very abstract commands that have no corresponding action in the real world. The use of a direct interaction technique for this type of commands would lead to an unnatural mapping that has to be learnt by the user anyway. Instead, virtual menus could be used because these provide several advantages that make it easier for the user to issue abstract commands [Bowman ‘95][Bowman ‘96][Mine ‘95].
Because menus display the options that are available to the user simultaneously, the user doesn’t have to remember them. If the number of options is limited to a well-defined set, they can help the user in making an appropriate selection. Furthermore, if the menu has a consistent, well-designed structure, the user can quickly learn how he can interact with the application. A good example of this is the use of a hierarchical menu structure if the number of options is too large to fit into one menu. This will take the user longer to reach commands that are deep in the hierarchy, but shortcuts (or accelerators) can be provided to solve this [Jacoby ‘92].
Besides allowing the user to give commands, menus also provide the user with information. Especially when context-sensitive menus are used, displaying the available options tells the user something about the current state of the application. Context-sensitive menus change their contents (i.e. the available options) based on the previous operatios that the user has performed. This provides the user with feedback on his actions, which is very important (see chapter 1) [Jacoby ‘92].
Another advantage of menus is that they allow the user to input only a few discrete values into the system, as opposed to the continuous nature of direct manipulation. An example of this is that instead of dragging an object to a particular location by grabbing it and moving it to the new location the user can select a menu item to move the object to one of a few possible locations. This inherent constraint improves accuracy with the user (although it may be too limiting in certain cases) [Bowman ‘95].
2 Implementation of virtual menus
While virtual menus have a lot in common with conventional menus, they also have some characteristics that introduce new problems in their design and use [Jacoby ‘92].
1 Placement of menus
The main difference between conventional menus and virtual menus is the fact that virtual menus exist in a three-dimensional environment, whereas conventional menus are displayed on a two-dimensional screen. Therefore, in the most general case, virtual menus may be at any position and orientation relative to the user. This can make these menus hard to read because objects may appear between the user and the menu, the user may be at a bad angle relative to the menu, or he may be too far or too near to the menu. Furthermore, if the menu is just another virtual object at a fixed location in the environment, the user can easily lose track of it [Jacoby ‘92].
An alternative approach is to constrain the menu to move with the user so that it remains visible at a fixed location in the user’s field-of-view. Here, the user won’t run the risk of not being able to find the menu, but the menu will obscure part of the user’s view of the scene. When the menu is fixed in the environment, the user can always look around in the scene without the menu blocking his view [Mine ‘95][Mine ‘97a]. A hybrid approach would be to allow the user to move around while the menu stays fixed in the environment. The user can then press a button (or perform a similar action) to snap the menu back into view [Mine ‘95].
Another possibility is to make the menu invisible and let the user perform a certain action to display the menu. In [Mine ‘97a] an interesting scheme is described where the menus are placed just outside the user’s field-of-view. The user can now physically reach up with his arm and pull down a menu. After interacting with the menu, the user just releases the menu which causes it to roll up out of view again. This technique takes advantage of the advantages of two-handed interaction as well (see chapter 5).
2 Dimensionality of virtual menus
Menus can be of differing dimensionalities. The dimensionality of a menu determines how many parameters the user has to change to select a particular menu item. It is important to note that it has nothing to do with the dimensionality of the object that represents the menu. In three-dimensional virtual environments menus can be one-, two-, or three-dimensional [Mine ‘95].
[pic]
Figure 6.1 Rotary tool chooser [WebCHIMP]
1 One-dimensional menus
One-dimensional menus are ideal for selecting one option from a limited set. The user can step through all available options one by one, for example by pressing a button. An example of a one-dimensional menu is the rotary chooser tool shown in Figure 6.1. The menu items are arranged in a semi-circle in the user’s field-of-view, and the user can select a particular menu item by rotating his hand [Mine ‘97b]. Note that, although the menu is one-dimensional, the individual menu items are represented by three-dimensional icons. One-dimensional menus are highly constrained, and therefore very easy to use.
2 Two-dimensional menus
Two-dimensional menus are similar to conventional menus used in desktop computers. The user can move a cursor in the two-dimensional plane of the menu to select an item. Since pull-down menus, control panels, and dialog boxes are two-dimensional entities, being able to look at it from all sides doesn’t improve its usability (rather, it makes it harder to use them), so two-dimensional menus should be used here [Bowman ‘95].
3 Three-dimensional menus
Adding a third dimension to the menu can increase power and flexibility, but it requires the user to specify all three dimensions which is very difficult in virtual environments. Basically, three-dimensional menus are the widgets that were discussed in chapter 5. As was argued above, when conventional menus (like pull-down menus) are used, three-dimensional interaction should be avoided.
3 The difficulty of selecting in VEs
As was discussed in chapter 4, pointing and selection are very difficult tasks in virtual environments, due to limitations of the hardware and of the user’s precision. It should come as no surprise that selecting menu items is just as hard. However, there are some ways to make this task easier for the user.
First of all, the user should be provided with sufficient feedback. When a cursor is displayed on the menu, this will help the user a great deal in determining which way to move to reach a particular option. Furthermore, the menu that is currently selectable should be highlighted, for example by displaying it with a distinctive color, or, when icons are used, by displaying the bounding box of the icon [Mine ‘95][Jacoby ‘92].
A further enhancement would be to use a sticky cursor. This still allows the user to move the cursor around freely, but the current menu item is highlighted until the cursor reaches the middle of the next item. Only then is this next item highlighted. Another improvement would be to provide hot areas. When the user highlights a menu item that is a hot area, this item is automatically selected, without requiring the user to explicitly select it. This is particularly useful for showing submenus [Jacoby ‘92].
Usually, the hands are used for selecting menu items. However, the hands are already being used for a lot of tasks in virtual environments. For example, when the user is manipulating a virtual object, it might be awkward for the user to be required to put the object down in order to select a command from a menu. In [Darken ‘94], an alternative approach is described in which the user selects menu items by saying the name of the command, which is a very natural way to give commands. This approach leaves the users hands free to continue working on other tasks. Furthermore, because the voice commands are available all the time, shortcuts are automatically provided. Experienced users don’t have to go through the entire menu structure to reach a particular command.
Finally, the advantages of two-handed manipulation discussed in chapter 5, may be used with menus as well. The user could hold the menu in his non-dominant hand, thereby providing a reference frame for his dominant hand which performs the actual interaction with the menu [Jacoby ‘92].
3 Virtual menus in CHIMP and the Virtual Gorilla Exhibit
Virtual menus have been used extensively in CHIMP and the Virtual Gorilla Exhibit. The various types will be described in this section.
1 CHIMP
In CHIMP, look-at menus and control panels are available to the user [Mine ‘96].
1 Look-at menus
CHIMP makes use of so-called look-at menus. These menus are distributed throughout the environment so that they are associated with the objects they control. Of course, this can make them hard to find. A look-at menu consists of a pop-up menu that is invisible by default, and a hot point (represented by a red sphere). Instead of text, icons are used to represent the menu options.
The user can activate a particular look-at menu by making the hot point fall within a hot zone. This hot zone is a circle that is centered in the user’s field-of-view, as illustrated in Figure 6.2. After this, the user can press and hold a button to make the pop-up menu visible. Which item is consequently highlighted depends on the current gaze direction of the user (hence the name). A highlighted item is selected by releasing the button.
[pic]
Figure 6.2 Look-at menu [WebCHIMP]
[pic]
Figure 6.3 Control panel [WebCHIMP]
The look-at menus in CHIMP can be used for:
• changing the current center-of-interaction (interaction relative to the user’s hand position or relative to the center of the object).
• selecting the current interaction tool (e.g. switching between manipulation and flying).
• Invoking various control panels.
2 Control panels
CHIMP makes extensive use of virtual control panels. These are the VE equivalents of standard dialog boxes. They can contain interface elements like sliders and buttons. A picture of such a control panel is shown in Figure 6.3. The menu is held in the user’s non-dominant hand, and the actual interaction is performed with the dominant hand. The user can select the various interface elements through the use of ray that shoots out of the user’s hand.
In CHIMP, three control panels are provided:
• Environment control panel
• Object control panel
• Tear-off object palette
[pic]
Figure 6.4 Tear-off palette [WebCHIMP]
[pic]
Figure 6.5 Alphanumeric input [WebCHIMP]
The environment control panel can be used for high-level editing functions (cut, copy, paste), functions to reset environment state, user position, and object scale, and undo/redo functions. The object control displays information about the currently selected objects that can subsequently be changed (e.g. object scale, position, and orientation). The tear-off object palette contains one icon for each object that can be placed in the scene. The user can place a new object in the scene by simply dragging a new copy off the palette into the environment (see Figure 6.4).
Finally, the control panels can be used to input alphanumeric data, which is usually a problem in virtual environments. This can be important for precise object placement. A ray is again used to point at a particular digit in a number, or an empty spot where the user wants to add a digit. When the user presses and holds a button, a pop-up list will appear consisting of the digits 0 through 9. The user can then select the desired digit. This is illustrated in Figure 6.5.
3 Virtual Gorilla Exhibit
In the Virtual Gorilla Exhibit the tablet that was described in the previous chapters serves as a virtual menu as well [WebGORILLADESIGN]. To the right of the map of the environment, a few icons are provided as menu items that allow the user to change interaction mode by selecting them with the stylus. Just like the control panels in CHIMP, the tablet is held in the user’s non-dominant hand. Interaction with the (two-dimensional) menu is improved by providing the user with a physical tablet. A picture of the virtual tablet was shown in paragraph 3.4.2.
7 Discussion and conclusions
In this final chapter the various techniques described in the previous chapters will be discussed and compared. To finish off the chapter, some conclusions will be drawn about interaction in virtual environments.
1 Virtual navigation
In chapter 3 several techniques for specifying direction and speed of movement were described. Physical motion is probably the most natural and intuitive technique. However, for most applications it is too constrained.
Hand directed techniques on the other hand have no inherent constraints. A general problem with hand directed techniques is the fact that when the user is navigating, he or she can’t use his hands for other tasks. Therefore, the user has to pause a task in progress (which involves his hands) when he or she wants to navigate, and vice versa. This isn’t very natural, and it may have a negative effect on performance of tasks that require a certain dexterity [Kheddar ‘95].
Gaze directed techniques can solve this problem, but the problem of the lack of inherent constraints remains. Therefore, it is important that virtual constraints are added. An example of this is that the position of the user’s head is constrained to be at a particular height above the virtual terrain, as was done in the Virtual Gorilla Exhibit. Another solution would be to provide the user with a more constrained technique in addition to unconstrained flying [Bowman ‘95].
The other techniques like dynamic scaling and using a World In Miniature allow for very fast navigation through the entire environment. However, it should be made sure that the user is able to maintain situation awareness.
A thing worth mentioning is the fact that the user generally has to have some means to initiate and stop movement. With physical motion this is trivial. All that is required here is a tracker to keep track of the position (and orientation) of the user. However, with hand and gaze directed techniques the user will probably have to press a button or perform a similar action (e.g. give speech commands) to start and stop navigating.
2 Virtual selection
In chapter 4 several different approaches have been described that allow the user to select virtual objects. Physical intersection techniques require the user to be within reach of an object and are therefore often too restricted. To solve this problem, ray casting and arm extension techniques were introduced. Arm extension technques are more natural than ray casting techniques (they imitate real world behavior, and generally use the same method for selecting both nearby and far-away objects), However, they are generally more difficult to use because the user’s virtual hand actually has to be inside an object to be able to select it, whereas ray casting techniques only require the ray to intersect with the object. Also, arm extension techniques still have a limited range depending on the length of the user’s arm. This can be accounted for by reducing the naturalness of the technique (e.g. stretch go-go).
A problem that remains with almost all of the described techniques is the difficulty to select small, distant objects, plus the fact that the target object has to be visible. Only indirect techniques do not suffer from these problems, but these require the user to remember the names of all objects. Another problem is that most techniques require the use of the arm, and may therefore lead to arm fatigue. Indirect techniques and gaze directed selecting do not have this problem.
The described techniques generally have been developed for the selection of just one object at a time. Selecting multiple objects would of course be possible by repeatedly using one of the described techniques until all desired objects have been selected. One technique that can easily be extended to simulateneously select multiple objects is the Framing hands technique described in paragraph 4.2.6. Here the user can resize the frame formed by his hands until it surrounds all desired objects [Pierce ‘97].
3 Virtual manipulation
Precise manipulation in virtual environments was shown to be very difficult in chapter 5. Direct manipulation, which is the most natural method, is complicated by the lack of constraints and specifically the lack of haptic feedback in VEs.
In the real world, people depend upon various constraints (gravity, solid objects, tabletops) when manipulating objects. Therefore, it is important to incorporate at least some of these constraints in the manipulation techniques that are available to the user. Some examples of this were described for CHIMP and the Virtual Gorilla Exhibit. Haptic hardware devices that are in existence nowadays are rather limited in that they only provide one finger or at most the hand with touch and force feedback. However, the use of physical props can often provide the user with important haptic cues as well. Various software techniques, like precise collision detection and simulated gravity can improve the situation as well.
4 Virtual menus
It is sometimes argued that the use of virtual menus (chapter 6) is undesirable because they represent an unnatural way of interacting with the environment. However, the user will always have to give abstract commands to the system. Because of the structured way in which menus present available commands to the user, they can be very effective for this type of commands and thereby improve performance.
The use of virtual menus suffers from the general difficulty of selecting and pointing tasks in virtual environments. In contrast with convential menus, virtual menus exist in a three-dimensional space. This may result in the menu items being very difficult to read. Because of this, it is generally preferable to use icons or other graphics instead of text to represent the menu items in virtual environmetns. These problems make it important to highly constrain the user’s interaction with the menu. In this chapter several methods have been described that accomplish this, such as one-dimensional menus, or the use of a physical prop like the tablet in the Virtual Gorilla Exhibit.
Finally, virtual menus provide a good means to give a particular command to the system with one hundred percent accuracy because of their discrete nature. They also allow the input of alphanumeric data, which is generally very difficult in virtual environments.
5 Conclusions
The large number and variety of the techniques described in this report indicates that a unifying framework for interaction in virtual environment is not to be expected anytime soon, if such a framework is possible at all. Each VE application places very different demands on which tasks should be available to the user and how he should perform them. In a training situation, the interaction techniques should mimic the real world environment in which the task later has to be performed as closely as possible. Data visualization applications, on the other hand, may require less natural techniques that allow quick viewing, selection, and manipulation of the data to gain valuable information. For each application a thorough evaluation should be made to select the interaction techniques that are most likely to improve user performance by allowing him to perform his tasks efficiently.
Because of the lack of constraints and technological limitations, natural interaction in three-dimensional virtual environments is very hard. Some promising haptic devices are being developed, but until the field of haptics matures, other methods have to be used to improve this situation. Physical props or virtual constraints are good candidated for this.
8 References
1 Books and papers
[Allison ‘96] D. Allison, B. Wills, L.F. Hodges, J. Wineman (1996). “Gorillas in the Bits.” Georgia Institute of Technology, Technical Report GIT-GVU-96-16.
[Barfield ‘95] W. Barfield, T. Furness, eds. (1995). “Virtual Environments and Advanced Interface Design.” Oxford University Press. Chapters 11-13, pp. 437-541.
[Bowman ‘95] D.A. Bowman, L.F. Hodges (1995). “User Interface Constraints for Immersive Virtual Environment Applications.” Georgia Institute of Technology, Technical Report GIT-GVU-95-26.
[Bowman ‘96] D.A. Bowman, L.F. Hodges, J. Bolter (1996). “The Virtual Venue: User-Computer Interaction in Information-Rich Virtual Environments.” Georgia Institute of Technology, Technical Report GIT-GVU-96-22.
[Bowman ’97] D.A. Bowman, L.F. Hodges (1997). “An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments.” Proceedings of the ACM Symposium on Interactive 3D Graphics, pp. 35-38.
[Chapman ‘92] D. Chapman, C. Ware (1992). “Manipulating the Future: Predictor Based Feedback for Velocity Control in Virtual Environment Navigation.” 1992 Symposium on Interactive 3D Graphics, pp. 63-66.
[Cutler ’97] L.D. Cutler, B. Fröhlich, P. Hanrahan (1997). “Two-Handed Direct Manipulation on the Responsive Workbench.” 1997 Symposium on Interactive 3D Graphics.
[Dani ‘96] T.H. Dani, R. Gadh (1996). “Creation of Concept Shape Designs via a Virtual Reality Interface.” Dept. of Mechanical Engineering, University of Wisconsin
[Darken ‘93] R.P. Darken, J.L. Sibert (1993). “A Toolset for Navigation in Virtual Environments.” Proceedings of User Interface Software and Technology ‘93, ACM SIGGRAPH, pp. 157-165.
[Darken ‘94] R. Darken (1994). “Hands-Off Interaction with Menus in Virtual Spaces.” Proceedings of SPIE 1994, Stereoscopic Displays and Virtual Reality Systems, Vol. 2177, pp. 365-371.
[Darken ‘96] R.P. Darken, J.L. Sibert (1996). “Navigating Large Virtual Spaces.” International Journal of Human-Computer Interaction, January-March 1996, 8(1), pp. 49-72
[Darken ‘97] R.P. Darken, W.R. Cockayne, D. Carmein (1997). “The Omni-Directional Treadmill: A Locomotion Device for Virtual Worlds.” Proceedings of User Interface Software and Technology ‘97, ACM SIGGRAPH, pp. 213-221.
[Figueiredo ‘93] Mauro Figueiredo, Klaus Böhm, José Teixeira (1993). “Advanced Interaction Techniques in Virtual Environments.” Computers & Graphics, Vol. 17, No. 6, pp. 665-661.
[Hinckley ‘94] K. Hinckley, R. Pausch, J. Goble, N. Kassell (1994). “Design Hints for Spatial Input.” Proceedings of ACM Symposium on User Interface Software and Technology, pp. 213-222.
[Jacoby ‘92] R. Jacoby, S. Ellis (1992). “Using Virtual Menus in a Virtual Environment.” Proceedings SPIE, Visual Data Interpretation, Vol. 1668, pp. 39-48.
[Kheddar ‘95] A. Kheddar, R. Chellali, P. Coiffet (1995). “Implementation of Head-Behaviour Based Control for Navigation Within Virtual Reality Applications.” IEEE International Conference on Systems, Man and Cybernetics: Intelligent Systems for the 21st Century, pp. 4644-4649.
[Kijima ‘96] R. Kijima, M. Hirose (1996). “Representative Spherical Method and Composition of Object Manipulation Methods.” Virtual Reality Annual International Symposium, VRAIS ’96, IEEE Computer Society Press, pp. 195-204
[Koller ‘96] D.R. Koller, M.R. Mine, S.E. Hudson (1996). “Head-Tracked Orbital Viewing: An Interactive Technique for Immersive Virtual Environments.” Proceedings of User Interface Software and Technology ’96, ACM SIGGRAPH, pp. 81-82.
[Mapes ‘95] D. Mapes, J. Moshell (1995). “A Two-Handed Interface for Object Manipulation in Virtual Environments.” Presence, Vol. 4, no. 4, Fall 1995, p. 403-416.
[Mine ‘95] M.R. Mine (1995). “Virtual Environment Interaction Techniques.” University of North Carolina, Technical Report TR95-018.
[Mine ‘96] M.R. Mine (1996). “Working in a Virtual World: Interaction Techniques Used in the Chapel Hill Immersive Modeling Program” University of North Carolina, Technical Report TR96-029.
[Mine ‘97a] M.R. Mine, F.P. Brooks Jr., C.H. Sequin (1997). “Moving Objects In Space: Exploiting Proprioception In Virtual-Environment Interaction.” Computer Graphics Proceedings, ACM SIGGRAPH 1997, pp. 19-26.
[Mine ‘97b] M.R. Mine (1997). “ISAAC: A Meta-CAD System for Virtual Environments.” Computer-Aided Design, Vol. 29, No. 8, pp. 547-553.
[Pausch ‘95] R. Pausch, T. Burnette, D. Brockway, M. Weiblen (1995). “Navigation and Locomotion in Virtual Worlds via Flight into Hand-Held Miniatures.” Proceedings SIGGRAPH ’95, in Computer Graphics, pp. 399-400.
[Pierce ‘97] J.S. Pierce, A. Forsberg, M.J. Conway, S. Hong, R. Zeleznik, M.R. Mine (1997). “Image Plane Interaction Techniques in 3D Immersive Environments.” 1997 Symposium on Interactive 3D Graphics.
[Poupyrev ‘96] I. Poupyrev, S. Weghorst, M. Billinghurst, T. Ichikawa (1996). “The Go-Go Interaction Technique: Non-Linear Mapping for Direct Manipulation in VR.” Proceedings of the ACM Symposium on User Interface Software and Technology, pp 79-80.
[Poupyrev ‘97] I. Poupyrev, S. Weghorst, M. Billinghurst, T. Ichikawa (1997). “A Framework and Testbed for Studying Manipulation Techniques for Immersive VR.” Proceedings of ACM Virtual Reality System and Technology Conference 1997, pp. 21-28.
[Rygol ‘95] M. Rygol, A. Dorbie, A. Worden, S. Ghee, C. Grimsdale, J. Harvey (1995). “Tools and Metaphors for User Interaction in Virtual Environments.” Virtual Reality Applications, R. Earnshaw, J. Vince, H. Jones, eds., Academic Press Ltd., pp. 149-161.
[Slater ‘95a] M. Slater, M. Usoh, A. Steed (1995). “Taking Steps: The Influence of a Walking Technique on Presence in Virtual Reality.” Transactions on Computer-Human Interaction, Vol. 2, No. 3, September 1995, pp. 201-219.
[Slater ‘95b] M. Slater, A. Steed, M. Usoh (1995). “The Virtual Treadmill: A Naturalistic Metaphor for Navigation in Immersive Virtual Environments.” First Eurographics Workshop on Virtual Reality, ed. By M. Goebel, pp. 71-86.
[Song ‘93] D. Song, M. Norman (1993). “Nonlinear Interactive Motion Control Techniques for Virtual Space Navigation.” IEEE 1993 Virtual Reality Annual International Symposium, VRAIS ’93, pp. 111-117.
[Stoakley ‘95] R. Stoakley, M. Conway, R. Pausch (1995). “Virtual Reality on a WIM: Interactive Worlds in Miniature.” Proceedings ACM SIGCHI Human Factors in Computer Systems, pp. 265-272.
[Van Loo ‘97] J.D. van Loo (1997). “Scientific Visualization in Virtual Environments.” TNO report
FEL-97-5040
[Vince ’95] J. Vince (1995). “Virtual Reality Systems.” Addison-Wesly Pub. Co.
ISBN 0-201-87687-6.
[Zhai ‘94] S. Zhai, W. Buxton, P. Milgram (1994). “The ‘Silk Cursor’: Investigating Transparency for 3D Target Acquisition.” Proceedings of ACM Conference on Human Factors in Computer Systems, CHI ’94.
2 World Wide Web Pages
[WebCHIMP]
[WebEVL]
[WebGOGO]
[WebGOGO2]
[WebGORILLADESIGN]
[WebGORILLAS]
[WebGRABBING]
[WebHINCKLEY]
[WebLOUKA]
[WebMITAILAB]
[WebNCSA]
[WebSARA]
[WebSTANFORD]
[WebUDW]
[WebVRMAT]
[WebVRSYSTEMS]
[WebWIM]
-----------------------
Crosshairs mode
Pointing mode
Gaze directed flying
Direction of flight
Bottom visible
Top visible
Front visible
Up Translation Area
Dead Area
Front Translation Area
Down Translation Area
Back Translation Area
accelerate
constant
decelerate
max speed
min speed
3 ZONE
LINEAR[pic]
1) Select
2) Grab
4) Release
3) Manipulate
Object movement
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- cross selling techniques in banking
- composition techniques in writing
- suggestive selling techniques in restaurants
- upselling techniques in hotels
- questioning techniques in the classroom
- memory management techniques in os
- promotional techniques in marketing
- referencing techniques in academic writing
- techniques in advertising
- statistical techniques in research
- sampling techniques in research pdf
- quantitative techniques in management pdf