copyright © 2005, pearson education, inc. chapter 5 direct manipulation and virtual environment

Copyright © 2005, Pearson Education, Inc.

Chapter 5

Direct Manipulation and Virtual Environment


Introduction

• Positive feelings associated with good user interfaces: – Mastery of the interface (user in control)– Competence in performing tasks – Ease in learning the system originally and in

assimilating advanced features – Enjoyment in using the system – Eagerness to show the system off to novices – Desire to explore more powerful aspects of the

system


Examples of Direct-Manipulation Systems

(1) Command line vs. display editors and word processors

• Training times with display editors are much less than line editors

• Line editors are generally more flexible and powerful • By the early 1990s, the display editors, described as what

you see is what you get (WYSIWYG), had become standard for word processors.

• The advances of WYSIWYG word processors: – Display a full page of text

• showing 20 to 60 lines of text simultaneously gives the reader a clearer sense of context for each sentence, while permitting easier reading and scanning of the document.

• By contrast, working with the one-line view offered by line editors is like seeing the world through a narrow card board tube.



• The advances of WYSIWYG word processors (cont.): – Display of the document in the form that it will appear

when the final printing is done• Table, lists, page breaks, skipped lines, section headings,

centered text and figures can be viewed in their final form. – Show cursor action

• Seeing an arrow, underscore, or blinking box on the screen gives the operator a clear sense of where to focus attention and to apply action.

– Control cursor motion through physically obvious and intuitively natural means

• Arrow keys or cursor-motion devices – such as a mouse, or trackpad – provide natural physical mechanisms for moving the cursor.



• The advances of WYSIWYG word processors (cont.): – Use of labeled icon for actions

• Most word processors have labeled icons in a toolbar for frequent actions.

• These buttons act as a permanent menu-selection display to remind users of the features and to enable rapid selection.

– Display of the results of an action immediately • When users press a button to move the cursor or center text,

the results are shown immediately on the screen.• Deletions are apparent immediately: the character, word, or

line is erased, and the remaining text is rearranged.• In contrast, with line editors, user must issue print or display

commands to see the results of changes.



• The advances of WYSIWYG word processors (cont.): – Provide rapid response and display

• Most display editors operate at high speed; a full page of text appears in a fraction of a second.

• This high display rate, coupled with short response time, produces a satisfying sense of power and speed.

– Offer easily reversible actions • An alternative offered by many display editors is a simple

undo action to return the text to the state that is was in before the previous action.

• Easy reversibility reduces user anxiety about making a mistake or destroying the file.


Examples of Direct-Manipulation Systems (cont.)

Technologies that derive from the word processor:

• Integration of graphics, spreadsheets, animations, photographs, and so on is done in the body of a document.

• Desktop publication software produces sophisticated printed formats with multiple columns and allows output to high-resolution printers.– Multiple fonts and color permit preparation of high-

quality documents, newsletters, reports, newspapers, or books.

– Examples include Adobe PageMaker and QuarkXPress. • Slide-presentation software produces color text and

graphic layouts for use directly from the computer with a large-screen projector to allow animations.



Technologies that derive from the word processor (cont.):

• Hypermedia environments and the World Wide Web allow users to jump from one page or article to another with selectable buttons or embedded hot links.

• Improved macro facilities enable users to construct, save, and edit sequences of frequently used actions.

• Spell checker and thesaurus • Grammar checkers



(2) The VisiCalc spreadsheet (the first electronic spreadsheet) and its descendants

• VisiCalc users delighted in watching the program propagate changes across the screen. – “Instantly calculating electronic worksheet” that permitted computation

and immediate display of results across 254 rows and 63 columns

• Competitors to VisiCalc emerged quickly; they made attractive improvements to the user interface and expanded the tasks that were supported.– Lotus 1-2-3 dominated the market in the 1980s, but– the current leader is Microsoft’s Excel, which has numerous features

and specialized additions.– Excel and other modern spreadsheet programs offer numerous graphics

displays, multiple windows, statistical routines, and database access.– The huge numbers of features are invoked with menus or toolbars, and

extensibility is provided by powerful macro facilities.



(3) Video games • From PONG to Nintendo GameCube, Sony PlayStation

2, and Microsoft Xbox• Field of action is visual and compelling • Commands are physical actions whose results are

immediately shown on the screen • No syntax to remember • Most games continuously display a score• Educational game - Direct manipulation in SimCity (learn

about urban planning)



(4) Computer-aided design • Most computer-aided design (CAD) systems for automobiles,

electronic circuitry, aircraft, or mechanical engineering use principles of direct manipulation.

• Building and home architects now have powerful tools such as AutoCad with components to handle structural engineering, floorplans, interiors, landscaping, plumbing, electrical installation, and much more.

• With programs like this, the operator may see a circuit schematic on the screen and, with mouse clicks be able to move components into or out of the proposed circuit.

• Similarly, newspaper-layout artists or automobilebody designers can easily try multiple designs in minutes (generate alternatives easily).

• The basic strategy for this design is to eliminate the need for complex commands that the operator might only need to recall during a once-a-year emergency.



(5) Office automation• Xerox Star was a pioneer with sophisticated

formatting– Users could move (but not drag) a document icon to a

printer icon to generate a hardcopy printout.• Rapid and continuous graphical interaction• Microsoft Windows is a descendant

– The Microsoft Windows design is still a close relative of the Macintosh design, and both are candidates for substantial improvements in window management, with simplifications for novices and increased power for sophisticated users.


Discussion of Direct Manipulation

Problems with direct manipulation • Spatial or visual representations can be too spread out

– Visual representations are not necessarily an improvement over text, especially for blind or vision-impaired users who will need special software.

– Graphical user interfaces were a setback for vision-impaired users, who appreciated the simplicity of linear command languages.

• Designs may force valuable information off of the screen, requiring scrolling or multiple actions.– For experienced users, a tabular textual display of 50 document names may be

more appropriate than only 10 graphic document icons with the names abbreviated to fit the icon size.

• Users must learn the graphical representations – A graphic icon may be meaningful to the designer, but for users may requires

as much learning time as a word, or more.• The visual representation may be misleading • Typing commands with the keyboard may be faster

– For experienced typists, taking a hand off the keyboard to move a mouse or point with a finger may take time than typing the relevant command.


Discussion of Direct Manipulation (cont.)

• The attraction of direct manipulation is apparent in the enthusiasm of the users.

• Three principles of direct manipulation: – Continuous representation of the objects and

actions of interest – Physical actions or presses of labeled buttons

instead of complex syntax – Rapid incremental reversible operations whose

effect on the object of interest is immediately visible


• Using these three principles, it is possible to design systems that have these beneficial attributes:

– Novices learn quickly – Experts work rapidly – Intermittent users can retain concepts – Error messages are rarely needed – Users see if their actions are furthering their goals – Users experience less anxiety (action can be

reversed easily)– Users gain confidence and mastery (they are the

initiators of action, they feel in control)



• Direct manipulation is a human-computer interaction style that was defined by Ben Shneiderman and which involves continuous representation of objects of interest, and rapid, reversible, incremental actions and feedback.

• Having real-world metaphors for objects and actions can – make it easier for a user to learn and use an interface (some

might say that the interface is more natural or intuitive), and – rapid, incremental feedback allows a user to make fewer

errors and complete tasks in less time, – because they can see the results of an action before

completing the action. – An example of direct-manipulation is resizing a graphical

shape, such as a rectangle, by dragging its corners or edges with a mouse.


http://en.wikipedia.org/wiki/Human-computer_interaction

http://en.wikipedia.org/wiki/Ben_Shneiderman

http://en.wikipedia.org/wiki/Ben_Shneiderman


Visual Thinking and Icons• An icon is an image, picture, or symbol representing

a concept • Icon-specific guidelines

– Represent the object or action in a familiar manner – Limit the number of different icons – Make icons stand out from the background – Consider three-dimensional icons (they are eye-catching but

also can be distracting)– Ensure a selected icon is visible from unselected icons

(distinctive)– Design the movement animation (when dragging an icon,

the user might move the whole icon)


Visual Thinking and Icons (cont.)

• Icon-specific guidelines (cont.)– Add detailed information, such as

• shading to show the size of a file (larger shadow indicates larger file),

• Color to show the age of a document (older might be yellower or grayer)

– Explore combinations of icons to create new objects or actions, for example, dragging a document icon to a folder, trash can, outbox, or printer icon has great utility.

• Can a user set security levels by dragging a document or folder to a guard dog or police car icon?


3D Interfaces• Some designers dream about building interfaces that

approach the richness of three-dimensional (3D) reality.– They believe that the closer the interfaces are to the real world, the

easier usage will be.

• “Enhanced” interfaces, better than reality, can help reduce the limitations of the real-world, e.g., providing simultaneous views.– User can magically change colors or shapes, duplicate objects,

shrink/expand objects, group/ungroup components.

• Among the many innovations, there have been questionable 3D prototypes, such as– Digital libraries: showing books on shelves may be nice for browsing,

but it inhibits search and linking


3D Interfaces (cont.)



Features for effective 3D (checklist for designers):– Minimize the number of navigation steps for users to accomplish

their tasks.– Keep text readable.– Avoid unnecessary visual clutter, distraction, contrast shifts, and

reflections.– Simplify user movement. (avoid surprises like going through

walls)– Prevent errors.– Simplify object movement– Organize groups of items in aligned structures to allow rapid

visual search.



Guidelines for inclusion of enhanced 3D features:– Provide overviews so users can see the big picture– Offer X-ray vision so users can see into or beyond

objects.– Provide history keeping (recording, undoing, replaying,

editing)– Permit rich user actions on objects (save, copy,

annotate, share, send)– Enable remote collaboration (synchronous,

asynchronous)– Give users control over explanatory text and let users

select for details on demand. (pop-up, screen tips)



Guidelines for inclusion of enhanced 3D features (cont.):– Offer tools to select, mark, and measure.– Implement dynamic queries to rapidly filter out

unneeded items.– Support semantic zooming and movement (simple

action brings object front and center)– Allow multiple coordinated views (users can see data

in more than one arrangement at a time)– Develop novel 3D icons to represent concepts that

are more recognizable and memorable.


Virtual and Augmented Reality

• Virtual reality or virtual environment breaks the physical limitations of space and allow users to act as though they were somewhere else – E.g. flight-simulator designers work hard to create the most realistic

experience for fighter or airline pilots.– 5DT (Fifth Dimension Technologies) is a high technology company

specializing in Virtual Reality (VR), see http://www.5dt.com/

• Augmented reality shows the real world with an overlay of additional overlay; for example,– While users are looking at the walls of a building, their

semitransparent eyeglasses show where the electrical wires or plumbing are located.

– Medical applications, such as allowing surgeons to look at a patient while they see an overlay of an x-ray to help locate a cancer tumor.

http://www.5dt.com/


Virtual and Augmented Reality (cont.)

• Successful virtual environments depend on the smooth integration of: – Visual Display

• The head-mounted displays block other images, so the effect is more dramatic, and head motion produces new images, so the users can get the impression of 360-degree coverage.

– Head position sensing • Head-mounted displays can provide differing views

depending on head position.• Look to the right, and you see a forest; look to the left, and

the forest gives way to a city.



The Head-mounted Display (HMD) offers the user affordable quality, high resolution (SVGA), a crisp image and superior sound quality packaged in a sleek, comfortable and extremely light headmount design.



• Successful virtual environments depend on the smooth integration of (cont.): – Hand-position sensing

• The DataGlove is an innovative invention that continues to be refined and improved beyond its current low resolution.

• Sensors for other body parts, such as knees, arms, or legs, are finding uses in sports training and movement capture.

The DataGlove is designed to satisfy the stringent requirements of modernMotion Capture and Animation Professionals.



• Successful virtual environments depend on the smooth integration of (cont.): – Force feedback

• Hand-operated remote-control devices for performing experiments in chemistry laboratories or for handling nuclear materials provide force feedback that gives users a good sense of when they grasp and object or bump into one.

– Sound input and output• Making convincing sounds at the correct moment with full

three-dimensional effect is possible, but it too is hard work.• The digital sound hardware is adequate, but the software

tools are still inadequate.

copyright © 2005, pearson education, inc. chapter 5 direct manipulation and virtual environment

Documents

line editors line editors

displaymost display

direct manipulation

system examples of direct

cursor action

high display rate

word processors training

lines of text