DESIGN OF INTERACTIVE SYSTEMS



DESIGN OF INTERACTIVE SYSTEMS

Overheads Set I

by Murray Turoff

Department of Computer and Information Science

New Jersey Institute of Technology

Newark NJ, 07102

TEL: 973 596 3399

email: turoff@vc.njit.edu

homepage://eies.njit.edu/~turoff/

© Copyright 1998 Murray Turoff

CONTENTS

Subject One: Fundamental Concepts, Observations and Problems

Subject Two: Dimensions of Interface Design

Subject Three: The Design Process

Subject Four: Protocol Analysis

Subject Five: Indexing & Searching

FUNDAMENTAL CONCEPTS

Subject One

© copyright 1998 Murray Turoff

Where is the wisdom we lost in knowledge?

Where is the knowledge we lost in information?

T. S. Eliot (The Rock)

ATMOSPHERE

HUMAN COMPUTER

Slow Rapid

Sloppy Rigorous

Forgetful Precise

Implicit Explicit

Subjective Objective

Brilliant Stupid

How do we design a communication process between the two?

WHY INTERACTIVE I

Iterative Problem Solving

Unpredictable Sequences

Tool Flexibility

Impossible Manually

Creativity and Quality

Collaboration, Coordination & Communications

WHY INTERACTIVE II

Enhancement (save time and/or effort)

Speed, quantity, memory

Enjoyment & Challenge

Sublimating & Escape

Power & Control

Solving the Wrong Problem

INFORMATION DOMAINS OF USERS I

Single Function Tasks

simple inquiry / calculations / messaging

Structuring Tasks

Organizing / filtering / summarizing

Status Briefing / Report Generation

Tracking / Monitoring Tasks

Creation / Composition / Modeling

INFORMATION DOMAINS OF USERS II

Challenges for Individual Support

Exception Reporting

Diagnosis

Discovery

Hypothesis Testing and Analysis

Challenges for Group Support

Planning, Decision Analysis and Resource Allocation

Command, Control & Coordination

STAGES OF USER EVOLUTION

Uncertainty Phase

Insight Phase

Incorporation Phase

Saturation Phase

USER CHARACTERISTICS

High Cognitive Variability, Problem Solvers, Creative, Demanding

Novice, Casual, Intermediaries, Operators, Routine

Experienced, Frequent, Power

Children to Adults

Clerk, Analyst, Engineer, Scientist, Manager, Executive

Results:

Different User Roles in one System

Multiple Interface Methods Required

DESIGN METHODS I

Comparison/Differentiating

Designing / Requirements Analysis

Task / Macro

Cognitive / Micro

Group / Micro & Macro

Process Design / Integration

Enterprise Design/Organizational

Technology / Environmental

DESIGN METHODS II

Enhancements / Evolution

Visioning / Normative Planning

Social Engineering

Goal Analysis

Enterprise Design

THE DEVELOPMENT PROCESS

[pic]

MODELS I

Mental Models of users resulting from their “real” world experiences

Requirements Model determined by the system analysis process acting on user mental models.

Implementation Model is the actual internal design model of the supporting software processes.

MODELS II

Real World is the actual system being model.

Interface Model/Metaphors is the mediator between the users mental model and the systems implementation model.

PROCESSES I

Comparison of the Implementation model to the

Requirements model is TESTING

Mental Model is EVALUATION

Real World is VALIDATION

Interface Model is TRAINING/LEARNING

PROCESSES II

Functional Opacity is created by large differences between the Interface Model and the Mental Model

System Opacity is created by large differences between the Interface Model and the Implementation Model.

Extreme functional opacity or system opacity can lead to cognitive dissonance.

VIRTUALITY

Virtuality is the process whereby the implementation model in the computer replaces the real world model in a system made up in part of people.

Information Systems are usually design on a descriptive basis; however, they often become prescriptive.

Designers need to anticipate prescriptive impacts of their systems.

The design of an interactive system can be the design of a social system

GENERAL SYSTEMS VIEW

Interactive Systems are open, dynamic, and adaptive in the long run

They will be changed by the organizational/social environment and they will influence changes in those environments.

Law of Requisite Variety Applies: The inputs to a system must be as rich/variable as the outputs if one wishes to control the outputs of a system.

FUNDAMENTAL PROBLEM I

Proper Level of Tools / Objects / Data Structure

Too Micro / Low Level

Difficult to work with

More effort and time

Too Macro / High Level

Inflexible

Difficult to Change

FUNDAMENTAL PROBLEM II

Gulf of Execution

Translating goals, intentions to actions

Functional Opacity

Gulf of Evaluation

Translating output display to evaluation

Not System Opacity but related

Excessive trial and error

PLAYING MUSIC PROBLEM

What are the principle alternative ways to make music?

Designing a system any idiot can use and only idiots will use it!

Novice Friendly, Experienced Hostile

Two Separate Dimensions

Beginning users / Experienced users

Non Domain Experts / Domain Experts

COMMON PSYCHOLOGICAL EFFECTS

Fishbowl

Bully

Peephole

Concrete

Clutter (Confusion)

People Angst

Computer Angst

Rorschach Blot

PSYCHOLOGICAL ROLES (Rorschach Blots) OF COMPUTERS

Evaluator, Magician, Helper, Entertainer, Companion, Challenger, Foe, Mentor, Accomplice, Producer, Overseer, Dictator, Priest, Servant, Picky Parent, Unruly Child, God, etc.

Computers can be made to appear to be emotional and/or intelligent.

Providing computer literacy almost an ethical issue

USER RESPONSE TO INADEQUATE SYSTEM

DIS-USE: Turn to other sources

MIS-USE: Using inappropriate ways

PARTIAL USE: Use of wrong subset

DISTANT USE: Use of intermediary

MINIMUM USE: Recipe (IRS type) interaction

NON-USE: Avoiding the system

MODIFICATION OF TASK: Change the task to fit the system (solving wrong problem)

COMPENSATORY ACTIVITY: User has to do more

DIRECT PROGRAMMING: User modifies system/System encourages unique interfaces

Sources of Knowledge and Wisdom I

General Social Sciences Methods

Controlled Experiments

Focus Groups

Interview Methods

Participant Observation

Survey Methods

Longitudinal studies

Field Studies

Sources of Knowledge and Wisdom II

Industrial Engineering

Human Factor studies

Physiological response studies

"time and motion" studies (Method)

Human Monitoring (Method)

Psychology

Human cognitive processes

Human problem solving

Protocol Analysis (Method)

Human psychology and mental ability measurement

Sources of Knowledge and Wisdom III

Management Sciences

Cost/Benefit Analysis (Method)

Efficiency, Effectiveness, Quality, Opportunity, Regret, Productivity

Delphi (Method)

Perceptions and System Monitoring

Measures of user satisfaction, efficiency of use, quality and effectiveness of applications

Management/Operational practices

Performance of Systems/Technology

Sources of Knowledge and Wisdom IV

Sociology

Conceptions of technology

Social and organizational systems

Group processes and behavior

Environmental interaction

Information Science

Data base Technology

Cognitive use of data and information

Information Retrieval

Sources of Knowledge and Wisdom V

Anthropology

Understanding metaphors and culture (Method)

Cultural adaptation

Human learning process

Sources of Knowledge and Wisdom VI

Archaeology

"study of artifacts" (Method)

Tools and toolmaking

Philosophy

"virtual reality"

Design of Social Systems

Virtuality

Sources of Knowledge and Wisdom VII

Computer Science & Information Systems

Designer Wisdom

Expert Systems

Interface Design

Applications

Computer Mediated Communications

Systems Analysis & Requirements Development

Process Reengineering

Enterprise Design

DIMENSIONS OF INTERFACE DESIGN

Subject two

© copyright 1998 Murray Turoff

CRITERIA FOR FACTOR DIMENSIONS

Can it be perceived?

Can it be measured?

Reproducible (repeatable)

Reliable (valid measure)

Orthogonal (Fundamental/Unique)

Can it guide design?

Relate to Interface design choices

Can it evaluate design?

Used to compare different designs

Can it sensitize designers?

Make them better designers

BASIC CATEGORIES OF DIMENSIONS

Foundation Factors

Understanding & Ease of Learning

User Sense of Control

Effectiveness

Psychological & Sociological

Administrative & Management

FOUNDATION FACTORS

Timeliness (Objective)

Responsiveness (Cognitive)

Reliability

Accessibility / Convince

Efficiency / Least Effort

Security

Accuracy

Protection/Bullet-proofing

UNDERSTANDING / EASE OF LEARNING I

Guidance / Informativeness

Conciseness / Brevity

Clarity / Simplicity

Comprehension

Segmentation / Decomposition

UNDERSTANDING / EASE OF LEARNING II

Consistency

Retention

Specificity

Familiarity

Scaffoling

SENSE OF CONTROL I

Leverage / Modifiability

Manipulability

Closure / Confirmation / Notification

Feedback / Sense of causality (Synchronous)

Contextual Visibility

Multi-tasking

Process Control / escape / interrupt / restart

SENSE OF CONTROL II

Forgiveness

Transparency

Flexibility / Cognitive Adaptation

Predictability / Regularity

Tracking / Monitoring

Backtracking / Auditing / History

Forecasting / Anticipating / Alerting

Backup / Undo / Listing

EFFECTIVENESS I

Task Functionality

Generality

Matching

Completeness

Abstraction / Leverage

Adaptability / Manipulability

EFFECTIVENESS II

Integration / Connectivity

Resiliency / Robustness / Richness

Relevance of objects and functions

Precision (minimize non relevant material)

PSYCHOLOGICAL & SOCIOLOGICAL I

Ethical / Honest / Private

Aesthetic / Pleasing / Artful

Interesting / Challenging / Fun

Self Image Enhancement

Peer Relations / Status

Sense of Community

Humanization / Polite

PSYCHOLOGICAL & SOCIOLOGICAL II

Satisfaction

Self, System, Group / Organization, Task / Job

Motivation

Expectations / “Self full filling prophecy”

Perceived Utility

Change & Social Inertia

Feeling of Participation

ADMINISTRATIVE

Training / Documentation

Maintenance of System

Job Enhancement

Human Help & Contact

Organization Relationships

System Evolution / Modifiability

Evaluation / Feedback

Costs and Charging

History & Confidence

EXAMPLES OF CONFLICTS AND TRADEOFFS

Comprehension Segmentation

Consistency Efficiency

Consistency Least Effort

Conciseness Informative

Conciseness Closure

Resiliency Ease of Learning

Task Generality Task Matching

Specificity Familiarity

Leverage Manipulability

COMMON DESIGN FAULTS I

Poor Interactive Sequences

Longer to do than manual (e.g. reading)

No tolerance or respect for human error

No flexible parsing (rigid syntax)

Wrong functionality

Start-stop hassle / Interruption handling

COMMON DESIGN FAULTS II

Integration across functions

Poor documentation and help

Inconsistent Metaphors

Lack of flexibility

Applicability to broad range of tasks

COMMON DESIGN FAULTS III

Multiple approaches to a given task

Multiple ways of invoking tasks

Alternative sequencing of functions

Adapt to different user styles / user types

INTERACTION METHODS I

MENUS: comprehension

COMMANDS: flexibility

LISTS: leverage

FORMS: clarity

DIALOGUE: guidance

WINDOWS: multi-tasking

ICONS: object representation

GUI: special representations

DIRECT MANIPULATION: control

INTERACTION METHODS II

MACROS: leverage

MIMICKING / RECORDING: task matching

ANIMATION: event sequencing

MODELS: abstraction

LANGUAGES: semantic representations

SCRIPTING: leverage, task matching

VIRTUAL REALITIES: understanding

AI AND EXPERT SYSTEMS: efficiency

THE DESIGN PROCESS

Subject Three

© copyright 1998 Murray Turoff

DOING DESIGN

Any type of design component can be the stimulus for an idea

Design can occur in any order through the maze of component relationships

Any incident of a component can be the stimulus for the idea process

Design is a non linear thought process

DESIGN COMPONENTS I

Goals and Objectives

User Tasks & Requirements

System Metaphors

System Artifacts/tools

ISSUES

How to make the computer recipe system more valuable to the user than any of the regular physical systems he or she can use?

How to extend the concept of what is in a recipe to be more than what the user initially conceives it to be?

RECIPE SYSTEM OBJECTIVES I

The user should be able to handle thousands of recipes.

Provide ways of organizing the information that is superior to that provided by books and index card files.

Identify useful functionality that is easier to take advantage of on the computer.

RECIPE SYSTEM OBJECTIVES II

Insure that individuals can learn to use this system very easily and be able to accomplish frequently occurring operations with a minimum of effort.

Provide facilities for the user to be able to learn more about cooking.

USER TASK: search requirements

Meals and meal parts (e.g. breakfast, snack, feast, main course, soup, desert, drinks, etc.)

Ingredients (meat, eggs, fruit, vegetables, etc.)

Nutritional (low fat, vegetarian, high protein, low salt, etc.)

Cultural (Chinese, various holidays, religious, ethnic, national, etc.)

Taste (hot, sour, sweet, bitter, coarse, fine, aromatic, fragrant, etc.)

RECIPE SYSTEM METAPHORS

Recipe, Meal

Cookbook

Card File

Scrap Book

Picture Metaphor

Dining Room, Kitchen, Root Cellar, Pantry, Menu (soup plate, etc.)

RECIPE TOOLS/ARTIFACTS

Nutritional analysis

e.g. calorie counter

Shopping list accumulation

User / Family Preferences Profile

e.g. Filters: likes, dislikes, necessary (low fat, diabetic, low salt)

Glossary, Historical Information

Food and meal layout function

TOOL EXAMPLE: RDA calibration

Function to modify Recommended Daily Allowance (RDA) values for individual (lower salt, etc.) which would change nutritional calculation

Total Fat, Saturated fat, Polyunsaturated fat, Monounsaturated fat, Cholesterol, Sodium, Carbohydrates, Dietary Fiber, Protein, Sugars, Vitamins, etc.

Would require a master ingredient list giving all such information per unit of ingredient.

DESIGN COMPONENTS II

Objects / sub-objects

Object Parts

Representations

Shortest: used to shorten lists

Abstract: provide overview

Content: to fill in total object

RECIPE OBJECT PARTS

"title"

Descriptors for indexing

Auxiliary information: e.g. number of servings, side dishes, drinks

List of "ingredients" and the "amount" of each

Set of cooking "instructions"

User notes, markups

Modification alternatives for special occasions, added by user

RECIPE REPRESENTATIONS

Shortest: Unique title

Abstract: title, index terms, auxiliary data

Content: ingredients, instructions, links

OTHER POTENTIAL OBJECTS

Meals

Key word index

Ingredients index

Marked list

MEAL/MENU Object

Classical Meal Object

Appetizers, Drinks, Breads, Soup, Salad, Cleanser, Main course(s), Vegetables, Starch, Desert, After dinner drinks

User should be able to define meal structure for different types of meals

dinner, lunch, breakfast, brunch, family style, picnic, etc.

Compilation of all ingredients for shopping list

DESIGN COMPONENTS III

Functions on objects

Generic (detail depends on object)

Explicit (self defined)

Strategic choice sets

Reactive choice sets

Controls

GENERIC & EXPLICIT FUNCTIONS

Generic:

Get, Find, Create, Modify, Delete

Explicit Commands for Index

Add, Remove (term), Index (as verb) object, Change (term), Merge (terms), Split (term), Order (index), vote

STRATEGIC MENU/CHOICES

|Object |Number | Actions |

|Recipes |3,455 | Get |

|Meals |238 | Find |

|Lists | | Changes |

| Keys |485 | Create / Add |

| Ingredients |9,989 | Modify /Change |

| Marked |35 | Delete / Remove |

EXAMPLE REACTIVE FUNCTIONS

Acting on some subpart of an object

Modify/add notes

Nutritional Analysis

Find Substitute/Alternative

Add to shopping list

REACTIVE MENU SAMPLE

Scan/List/View

Find

Mark/unmark

Add (index term/instruction/ingredient))

Remove (index term)

Controls: Home (escape), End (finish), Help, Show

EXAMPLES OF CONTROLS

Go (how far), back, forward, elsewhere, open/close/size windows, etc.

Print/File (how much): object, screen, page, list, window, link, etc.

Task control, confirm, quit, help, undo, escape, finish, interrupt, continue, copy, cut, paste, open, close, etc.

One can trace the evolution of interface design through the evolution as to what has become a commonly acceptable control “tool”

DESIGN COMPONENTS IV

Modifiers and Status states

Subsets, tracking

creates meaningful lists

Lateral semantic linkages

Shared processes

List Processing

Searching

MODIFIERS

Create subset/list of objects

Examples

Authored recipes

Incomplete recipes

Sent messages

New messages

LINKAGES I

Meal to recipes

Recipe to meals

Ingredient

as recipe

to details

recipes/meals utilizing it

to substitutions

Number of servings to change program

LINKAGES II

To added Information

cooking method

utensils

nutritional details

Name to similar recipes

Related items: e.g. wine, side dishes

Time to prepare, to cooking

LINKAGE TYPES: Open Ended

Two way usually desirable

Part of (recipe in meal)

List generation (all meals with recipe)

More than one link from object/anchor

Alternative/substitution (of ingredient)

Explanation/Information (about ingredient)

Association/equality (other recipe)

Trigger program (e.g. servings, nutrition)

OBSERVATION ABOUT RECIPE

Natural contextual menu

user knows context

user can be taught context

User can learn easily through trial and error

Ease of learning clear

LIST PROCESSING

Adding and removing: index terms, indexed objects

Dividing, Merging, or Sorting lists

Comparing lists (multiple object entries)

Inverting (all occurrences)

Expanding, Contracting levels

Marking

KEY LIST SAMPLE I

meat

veal

...

beef

sweetbreads

brain

red meat

roasts

...

KEY LIST SAMPLE II

roasts

London Broil

Eye

Rump

...

steak

...

processed

hot dogs

ALTERNATIVE KEY STRUCTURES

Turkey

Holidays

Thanksgiving

Christmas

Leftovers

...

Fowl

Turkey

White meat

Breast

KEY LIST OPTIONS

Meat levels: 9 Items: 455

Use + or ... or "number" to expand or contract

Allow objects in multiple locations

meat.beef.red_meat.steaks.skirt (Shows where it came from)

SEARCH ISSUES I

What specific reactive searches are needed throughout the interface

Initiating the search and being informed of the results with respect to the amount of material found.

SEARCH ISSUES II

Deciding and indicating what is to be retained or discarded in a search result.

Whether to initiate a search of material found or to search the material not retrieved and how to combine this with the prior searches.

DESIGN COMPONENTS V

Object Formats

Screen layouts

Workspace, status areas

Control areas, message areas

OBJECT FORMAT

Rattlesnake Stew

Created: 8/12/91 Modified: 6/5/92

Keys: /meat.snake/exotic/rich/

American.southwest.Indian/

Servings: 2 Preparation Time: 90 min.

Last used: 12/8/92 Times used: 15

Characteristics: High Fat, Stewed

Utensils: Simmering Pot

Remarks: Supplied by John Franklin from my National Guard unit. Tastes very much like rich oily chicken.

INTEGRATION EXTENSIONS

Electronic Shopping

Maintaining inventory

Exchange recipes with other users

Group Meal Planning

Education: Virtual World Recipe System

Being able to actually replicate the cooking process in the virtual world.

Being able to have an instructor join you in that environment.

Being able to cook as a group.

DESIGN COMPONENTS VI

Processes and Closure

Processes and events that take place independent of user.

Interaction process & modes

The states and modes a user can be in.

User Object lists

The lists of objects the user will need or desire to manipulate.

DESIGN COMPONENTS VII

Error conditions

Identification of possible errors that the user could make.

System Messages

Notifications, Closures, Confirmations, and Error Messages.

Necessary Help

Identification of what is unclear or not self evident in the interface.

DESIGN COMPONENTS VIII

For Specialized Systems

CMC: roles, notifications

DATABASES: forms, entity models, reports

COMMUNICATIONS: protocol scripts/templates

HYPERTEXT: anchor, linkage types

Design Levels I

|GENERAL |SPECIFIC |DETAIL |

|Goals |User Task | |

|Metaphors | | |

|Objects |Heading, Abstract |Object parts |

| |Content |Sub-objects |

| | |Formats |

| | |Linkages |

|Functions / Actions |Generic |Explicit, Controls |

| |Links | |

Design Levels 2

|GENERAL |SPECIFIC |DETAIL |

|Strategic Choice Sets |Reactive Choice Sets | |

|Shared Processes |Searching | |

| |List Processing | |

|Interaction States |Screen layouts |Workspace, Status, Control, & System |

| |Error Conditions |message area |

| |Necessary Help | |

PROTOCOL ANALYSIS

Subject Four

© copyright 1998 Murray Turoff

Language is the dress of thought

Samuel Johnson

WHY DO IT?

Objective: To discover the process a person goes through in solving a problem.

Uses: Learning Cognitive Processes, Developing Expert System material, evaluating interfaces.

KEY ASSUMPTION

Cognitive processes that generate verbalization are subset of those that generate behavior

Example: Lisa Learning, by Carroll and Mazur, IEEE Computer November 1986.

CONCERNS ABOUT THE METHOD

Subjects may have incomplete knowledge of their thinking processes; therefore the record may be incomplete.

Subjects may not have an accurate understanding of the processes of which they are aware.

The thinking process may be distorted by the thinking out loud process.

Ambiguity in language may lead to miscommunications.

NECESSARY ATMOSPHERE

Honesty

No evaluation of subject

No pressure for performance

No introduction of bias

No contamination of mental process (e.g. help)

Reciprocity and Respect

STIMULUS RESPONSE METHOD

Different stimulus may produce different mental behaviors

Do you know the capital of Sweden?

Which of the three: Oslo, Stockholm, or Copenhagen is the capital?

Name the capital of Sweden.

How much does it cost to fly and elephant from Thailand to Los Angeles?

How long is the Nile river?

MODES OF PROBING

Talk Aloud, Think Aloud: While information is attended.

Concurrent Probing: While in short term memory.

Retrospective Probing: After completion of the task.

PROTOCOL ANALYSIS CONDITIONS

Subject asked to verbalize what they are thinking

Subject is not being evaluated

Observer must not participate in process

Observer must not aid the subject

Subject providing knowledge of how they solve or a problem (or learn a system)

PROTOCOL ANALYSIS PROCESS I

Present the subject a written explanation of what is taking place.

Explain that this is to evaluate the system and not them

Explain you are there only to observe and can not help them.

Present in writing a task written the user terminology and the way the user would think about it.

PROTOCOL ANALYSIS PROCESS II

Total time should take between thirty minutes to an hour if no major problem encountered.

Observe and record (video, voice tape, PC interactions, notes and coding)

Only interrupt user for further verbalization if it is unclear what they are doing.

PROTOCOL ANALYSIS PROCESS III

Ask the user to describe what he or she is doing outloud.

Ask the user to go through all the terms on the screen and explain what they think they mean

Ask the user to forecast what they think an action will do

Carry out the task on the system

PROTOCOL ANALYSIS PROCESS IV

Give help only if user is at a dead end

Questions you can ask during if necessary

Why do you do/say that?

What is troubling you?

How do you know that ...?

Why do you do it this way?

Save retrospective questions for end of session.

MAJOR LIMITATION

Can not use it on a task that the user is not familiar with.

Requires training on the task first.

TYPICAL WRITTEN INSTRUCTIONS

PLEASE EXPLAIN:

What you think something means.

What you are trying to do.

What confusion or concerns you have.

What you expect to happen next.

What you don’t know the meaning of.

CODING SCHEMA FOR VERBALIZATION I

EXAMPLE I:

Intentions: goals, shall, will, must, have to

Cognitions: current attention situation, define, understand

Planning: If x than y

Evaluation: Yes, No, Damit, Fine

Changing conditions \ view of the problem

CODING SCHEMA FOR VERBALIZATION II

EXAMPLE II:

Surveying given information

Generating new information

Developing a hypothesis

Unsuccessful solutions

Self reference or criticism

Silence

PROTOCOL ANALYSIS OBSERVATIONS I

Verbalization occurs only 30% to 50% of the time.

Subjects cannot verbalize when:

Reading text

Doing intense cognitive activity

Making choices

Subjects have to slow down to verbalize

Subjects will tend to be more careful as a result of verbalization and observation

PROTOCOL ANALYSIS OBSERVATIONS II

Experts on a problem verbalize a lot more than non experts (double)

Experts have more difficulty verbalizing at a very detailed level with respect to the problem with which they are dealing with.

Experts usually spend more time in planning and in the restructuring of the problem.

INCREASING VERBALIZATION

1. Hold back stimulus or encourage slowness.

2. Segment stimulus (subtasks)

3. Interrupt with pre-arranged signal or set point to pause

e.g. when you are ready to indicate an action, first explain what you think everything means on the screen.

OBJECTIVES FOR INTERACTIVE SYSTEMS I

Determine their understanding of terms in the interface

Understand the cause of errors or misinterpretations

Determining missing functionality or user requirements

OBJECTIVES FOR INTERACTIVE SYSTEMS II

Determining reactions to and utility of features

Determine the utility of a metaphor for learning

Determining the utility of help and guidance

PROTOCOL ANALYSIS ADVANTAGES

A lot less effort than other approaches.

Can be done with prototype or mockup before any coding.

Learning how user approaches task

Finding major mistakes in design

Can learn attitude

Rapid feedback from small samples

Also useful for understanding user task

PROTOCOL ANALYSIS REQUIREMENTS

Subjects must be representative

Three subjects for each distinctive type of user and for each different set of tasks

Instructions simple

Observe only

PROTOCOL ANALYSIS QUESTIONS

Can ask/say:

Please explain your choice.

What are you thinking?

What does that term mean?

Should not ask:

Why did you do that?

What does “append” do?

Have physical signal for interrupt

PROTOCOL ANALYSIS: HOW TO I

A one page explanation to the subject

Set of written tasks in user terms

Subject should only spend 40-60 minutes.

Categorization scheme for recording

Tape record their verbalizations

PROTOCOL ANALYSIS HOW TO II

Retrospective questionnaire for end

Retention of major concepts

Perceived utility of features/functionality

Do not try to test everything

At least three subjects on same task

Be specific about user explaining choice they are about to make

UTILITY OF MOCK UP

Ease of understanding (current knowledge)

Ambiguity in terms

Confusion generation

Loss of continuity

Developing on line help

Marketing and acceptance

UTILITY OF WORKING SYSTEM

Ease of learning (new knowledge)

Error impact & Utility of feedback

Ease of exploration

Realistic task execution

Developing final user documentation

Utility of new features (beyond current system)

Wizard of Oz System

Allows simulation of working system

Dummy output data stored

Working choices for moving among screens and windows

More software to do this appearing on the market

BASIC QUESTIONS I

Do the terms used on the screen mean to the subject what the designer thought they would mean?

Do the alternatives presented at that point in the interaction include what the subject wishes to do?

Is the help material or the system messages understandable or relevant to the needs of the user?

BASIC QUESTIONS II

Does the subject have difficulty locating or perceiving things on the particular screen?

Does the subject utilize the sequences of operations that the designer expected to be utilized in accomplishing a given task?

Can the user utilize the interface metaphor for learning the system?

What type of errors is the user making and why?

WHEN TO USE

Protocol analysis should be used

Before system requirements are finalized

After every major change to requirements or interface

Before instillation of system for users

Before introduction to new user population type

INDEXING & SEARCHING

Subject Five

© copyright 1998 Murray Turoff

"When I use a word," Humpty-Dumpty said, "it means

just what I choose it to mean -- neither more nor less."

--Lewis Carroll, Alice in Wonderland

PART OF THE PROBLEM

Bible, Holy Bible, La Sainte Bible, Biblis Sacra, Bible wordo, biblia, Scriptures, Holy Scriptures, The Scriptures, Sacra Scrittur, Saintes, New Testament, Old Testament, Testament, Nouveau Testament, Avrcien, Vetus Testamentum, vetus novum, nuovo testament, gospels, evangelium secundum, mathhewm, gospel of St. Matthew, epistle to the romans, acts of the apostles, proverbs, psalms ecclesistes.

SEARCH MATCHING PROBLEM

IF DEALING WITH COLLABORATIVE INPUTS TO DATA

Air Force Base

air force base

AFB

A.F.B.

A. F. B.

MULTIPLE MEANINGS: AMBIGUITY

What is a “tank”?

weapon

oil storage tank

fish tank

part of toilet unit

May need context to resolve

Use of jargon and codes in groups

INDEXING TYPES

Hierarchical

Network (lateral)

Subject headings

Key words

Syntactic languages

Phrases

Natural Language

HIERARCHICAL

Single location in tree

Precise object (definition of item)

e.g. 1, 1.1, 1.2, 2.1, 2.1.1

e.g. outlines, MDSE. files

Minimum coding size, difficult to do and use

Rigid, difficult to evolve / adapt

NETWORK (LATERAL LINKS)

Single Location in network

Precise relationship (definition of link)

e.g. book index, citation index

e.g. Hypertext (in theory, not practice)

Lack of global view

CITATION INDEX I

Index specific to a time like last year or last five years

Source of data all professional papers published during that time frame (on line version for all years recorded).

Papers published in refereed journals and conferences

CITATION INDEX II

For a paper published at any time in the past and referenced by papers published in that time frame, the results is a list of which papers referenced the earlier papers.

Uses reference list at back of papers to create an “inverted index”

CITATION INDEX PROPERTIES

Most papers cease to be referenced five to ten years after publication

Most important contributions to science are papers reference ten and twenty years after publication

Some fields have significant number of papers never referenced (e.g. math is 30%)

CITATION INDEX UTILITY

Useful for finding recent work in a given field defined by important papers in the field.

Useful for evaluating the contribution of individual scientists over the long term.

Takes 3 to 5 years after publication to begin to assess importance of individual paper.

SUBJECT HEADINGS

Fixed categories, no structure, categorical scale

Precise definitions of bin in which to fit items: Problem: book on “Use of integral equations in Physics”

Open choice as to file objects in multiple headings but not commonly done except through lateral linkages (e.g. cross references)

KEY WORD AND COORDINATE SYSTEMS

Fixed key words

Free key words

Multiple keys for single object

Coordinates for properties

e.g. tall, medium, or short

SYNTACTIC LANGUAGES: Tagged Descriptors

Qualified keys, associated subcategories

Mix of key word with attached hierarchies

e.g. tank.weapon or tank.petroleum

e.g. steak.beef, steak.lamb, steak.veal

(type of meet).(source)

SYNTACTIC LANGUAGES: Faceted Index

Separate dimensions to represent item

Like the many facets of a diamond

softness, style, color, grade

reference: author, title, source

Very common in science and commerce

leather, wine, metal alloys, plastics

Both objective and subjective dimensions

MIXED INDEX EXAMPLE

UNIVERSAL DECIMAL CLASSIFICATION

341.67:623.454.8(094.2)

341.67: Disarmament, limitation and control of weapons of mass destruction

623.45: Ammunition, pyrotechnic devices, weapons of mass destruction

623.454.8: Active rays, atomic nuclear (thermo) weapons

094.2: Historical sources (09), International treaties

Why is nuclear weapons a subcategory of ammunition?

NATURAL LANGUAGE: Phrases

Short phrases, titles

chapter/book titles, paper titles

e.g. KWIC index

Often poor because done by original author

KWIC INDEX

Key Word In Context

Alphabetical listing of all titles permuted word by word

Words like of, the, a, an etc. not used.

Acceptance of Groupware Systems.

Groupware Systems. Acceptance of

Systems. Acceptance of Groupware

NATURAL LANGUAGE: Abstracts, Paragraphs

Long enough to distinguish the single item uniquely from all others in the collection of items

Good when searchers and writers of abstracts are experts in domain

English ambiguity a major problem

INDEXING EFFECTIVENESS I

Relevant Non Relevant

Retrieved A B

Not Retrieved C D

Precision = A / (A+B)

Recall = A / (A+C)

Specificity = D / (B+D)

Total Size Dependent (D)

Search Efficiency = (Recall)x(Specificity)

INDEXING EFFECTIVENESS II

Cannot have both high precision and high recall with qualitative information and even very complex quantitative databases (e.g. all known plastics, metal alloys, etc.)

Indexing methods represent compromise between these two opposing objectives.

INDEXING EFFECTIVENESS III

Too much recall leads to information overload and/or more effort for user

Too much precision leads to loss of relevant material and poor quality result for user

INDEXING EFFECTIVENESS IV

Timeliness (updating and currency)

Accuracy

Completeness (All data in database)

Form/structure of data useful (e.g. summary, raw data)

Depth and Breath (depth easier for novices, breath better for experts)

INDEXING EFFECTIVENESS V

Discrimination & Resolution / Density

Subjective / Objective

Adoption & Acceptance

Historical relevance

Integration (analysis tools, other sources, etc.)

EVALUATION DIMENSIONS I

INDEX TYPE

AMBIGUITY EXPRESSIVE CONCISE

HIERARCHICAL

LOW LOW HIGH

NETWORK

SUBJECTS

FIXED KEYS

FREE KEYS

TAGGED DESCRIPTORS

FACETED INDEXES

PHRASES

NATURAL LANGUAGE

HIGH HIGH LOW

EVALUATION DIMENSIONS II

INDEX TYPE

RETRIEVAL SELECTION ADOPTION

EFFORT EFFORT EFFORT

HIERARCHICAL

LOW HIGH HIGH

NETWORK

SUBJECTS

FIXED KEYS

FREE KEYS LOW

TAGGED DESCRIPTORS

FACETED INDEXES

PHRASES

NATURAL LANGUAGE

HIGH LOW HIGH

IDEAL USE OF INDEX TYPES I

HIERARCHICAL

Macro concepts, well structured area (table of contents)

NETWORK

Micro concepts, structured relationships (hypertext document)

SUBJECTS

Macro concepts, structured objects (professional papers)

IDEAL USE OF INDEX TYPES II

FIXED & FREE KEYS

Micro concepts, unstructured, evolving (book index, files)

Free if area is evolving

Fixed when creators know more than users

IDEAL USE OF INDEX TYPES III

TAGGED DESCRIPTORS & FACETED INDEXES

Micro, structured factors, well understood dimensions (e.g. materials, commodities)

PHRASES

Macro, semi-structured (message titles, paper titles, book titles, etc.)

NATURAL LANGUAGE

Macro, unstructured (abstracts)

ZIPF’S LAW

Log frequency of terms in the English language (Y axis) are linear with log of rank order (X axis).

Product of frequency and rank order raised to fixed power is a statistical constant

Non log plot often called “draw down curve” and used in other fields.

SAMPLE Zipf List I

Word Rank Frequency R x F

the 1 68,315 68,315

of 2 35,716 71,432

and 3 27,856 83,568

to 4 26,760 107,040

a 5 22,744 113,720

in 6 21,108 126,648

that 7 11,188 78,316

is 8 10,978 87,824

was 9 10,499 94,491

it 10 10,010 100,100

Sample Zipf List II

Word Rank Frequency R x F

sir 195 452 88,140

it's 196 452 88,592

why 197 451 88,847

asked 198 448 88,704

give 199 446 88,754

once 200 443 88,600

Sample Zipf list III

Word Rank Frequency R x F

usually 400 239 95,600

tax 500 167 83,500

ideas 800 128 102,400

proved 1,170 88 102,960

sections 2,146 49 105,154

flames 5,070 17 86,190

cultures 7,020 11 77,220

OBSERVATIONS ON LANGUAGE

Order: Easiest to say to hardest to say (amazing collective evolution of language)

Can be derived from Classical Information Theory by minimizing total length of codes to represent message. Done years later after Zipf’s empirical studies.

CONSEQUENCES

Led to Zipf’s Principle of Least Human Effort

Human Behavior and the principle of least human effort: an introduction to human ecology, George Kngsley Zipf, 1949, 1972.

Concepts extended to many other areas of human and biological activity.

SEMANTIC BALANCE I

Speaker’s economy in possessing a vocabulary of one word with m different meanings.

Auditor’s (listener) economy in possessing a vocabulary of m different words with one distinctive meaning.

Conflict in language and cause of “ambiguity.” A collection of m words will have more than m meanings.

SEMANTIC BALANCE II

First 1,000 words (after initial 500) have like 6,000 meanings, meaning they are used much more often than later words. (Ave. = 6)

First 20,000 words have like 40,000 meanings (Ave. = 2)

Suitable packet (unit of words) size can fit harmonic series

Compromise and tradeoffs in human affairs

Zipf RELATIONSHIPS

Form of equation from: Generalized Harmonic Series

r x f ^ (1/p) = C

r = rank f = frequency

1 >= p >=0

for English p ~ 1, but specialized vocabularies differ slightly.

Distribution of star types provides insight into evolution

ZIPF’s STUDIES I

Number of retail establishments vs. population

Diversity (number of different establishments) vc. population

Vocabulary of NY Times vs. ordinary paper (constant a lot larger)

Tool layout for craftsperson (furniture makers, shoe makers, watchmakers, etc.)

ZIPF’s STUDIES II

Frequency of marriage vs. distance in blocks (before cars)

Visitors to national parks vs. distance of home

Size of cities (derived from agricultural model) before last few decade

Number and diversity of establishments vs. size of population

Biomass (frequency x weight) in pond (closed ecological system) constant among species in food chain.

SYSTEMATIC KEY WORD LIST DEVELOPMENT I

Analyze body of literature (significant sample) in application domain

Compare order to normal English and highlight significant differences

Concordance study useful for any writers

High frequency useless: too many matches

SYSTEMATIC KEY WORD LIST DEVELOPMENT II

Low frequency useless: too few matches

Use middle frequency terms different from normal English

Do correlation studies of two, three, four, and five terms together (within same paragraph, sentence) to seek clusters of matches.

PREPARING THE SAMPLE

Remove useless words (“Exclusion list) from initial sample

Illustration of “a” words from such a list:

a about above after afterall afterward again against all almost already also although always am an and another any anyone anything are as at

Remove suffixes: (e.g. age als ant ary) from all words in sample

SEARCH APPROACHES I

Scanning: Broad, top level, term lists

Browsing: Up down, laterally, following interest, iterative, backup and start over

Target Searching: Specific target objective, need structure of data, categorization rules and understanding

SEARCH APPROACHES II

Exploring: Objective is learning the nature of the database and contents.

Wandering: Creative stimulus, random patterns

Both are a significant design challenge

SUBJECTIVE HUMAN INDEXING

Spatial Indexing (e.g. piles of paper)

Event Indexing (e.g. calendars)

Color & Form Indexing (e.g. tabs, marks)

Categorical (e.g. priorities)

SEARCH PROCEDURES I

1 Perceiving a need to find

information.

2 Formulating the request for

information.

3 Selection of the source for the

information.

SEARCH PROCEDURES II

4 Specifying the search strategy.

4.1 Browse the index, provide

numbers

4.2 Expand and contract levels

4.3 Mark multiple choices (or)

4.4 Provide for: or (novice), and, not

conditions, also distance

conditions

SEARCH PROCEDURES III

5 Carrying out the search

(mechanics).

5.1 Show number of hits

dynamically

5.2 Browse the hit list

5.3 Mark, Nest, Backup, Merge,

Restart

SEARCH PROCEDURES IV

6 Evaluating the results

6.1 Update index or list. Determine

relevance

6.2 Saving, printing, viewing,

categorization

6.3 Merging, reordering, clean up,

store strategy

End of Lecture 1-5 Set

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download