DESIGN OF INTERACTIVE SYSTEMS
DESIGN OF INTERACTIVE SYSTEMS
Overheads Set I
by Murray Turoff
Department of Computer and Information Science
New Jersey Institute of Technology
Newark NJ, 07102
TEL: 973 596 3399
email: turoff@vc.njit.edu
homepage://eies.njit.edu/~turoff/
© Copyright 1998 Murray Turoff
CONTENTS
Subject One: Fundamental Concepts, Observations and Problems
Subject Two: Dimensions of Interface Design
Subject Three: The Design Process
Subject Four: Protocol Analysis
Subject Five: Indexing & Searching
FUNDAMENTAL CONCEPTS
Subject One
© copyright 1998 Murray Turoff
Where is the wisdom we lost in knowledge?
Where is the knowledge we lost in information?
T. S. Eliot (The Rock)
ATMOSPHERE
HUMAN COMPUTER
Slow Rapid
Sloppy Rigorous
Forgetful Precise
Implicit Explicit
Subjective Objective
Brilliant Stupid
How do we design a communication process between the two?
WHY INTERACTIVE I
Iterative Problem Solving
Unpredictable Sequences
Tool Flexibility
Impossible Manually
Creativity and Quality
Collaboration, Coordination & Communications
WHY INTERACTIVE II
Enhancement (save time and/or effort)
Speed, quantity, memory
Enjoyment & Challenge
Sublimating & Escape
Power & Control
Solving the Wrong Problem
INFORMATION DOMAINS OF USERS I
Single Function Tasks
simple inquiry / calculations / messaging
Structuring Tasks
Organizing / filtering / summarizing
Status Briefing / Report Generation
Tracking / Monitoring Tasks
Creation / Composition / Modeling
INFORMATION DOMAINS OF USERS II
Challenges for Individual Support
Exception Reporting
Diagnosis
Discovery
Hypothesis Testing and Analysis
Challenges for Group Support
Planning, Decision Analysis and Resource Allocation
Command, Control & Coordination
STAGES OF USER EVOLUTION
Uncertainty Phase
Insight Phase
Incorporation Phase
Saturation Phase
USER CHARACTERISTICS
High Cognitive Variability, Problem Solvers, Creative, Demanding
Novice, Casual, Intermediaries, Operators, Routine
Experienced, Frequent, Power
Children to Adults
Clerk, Analyst, Engineer, Scientist, Manager, Executive
Results:
Different User Roles in one System
Multiple Interface Methods Required
DESIGN METHODS I
Comparison/Differentiating
Designing / Requirements Analysis
Task / Macro
Cognitive / Micro
Group / Micro & Macro
Process Design / Integration
Enterprise Design/Organizational
Technology / Environmental
DESIGN METHODS II
Enhancements / Evolution
Visioning / Normative Planning
Social Engineering
Goal Analysis
Enterprise Design
THE DEVELOPMENT PROCESS
[pic]
MODELS I
Mental Models of users resulting from their “real” world experiences
Requirements Model determined by the system analysis process acting on user mental models.
Implementation Model is the actual internal design model of the supporting software processes.
MODELS II
Real World is the actual system being model.
Interface Model/Metaphors is the mediator between the users mental model and the systems implementation model.
PROCESSES I
Comparison of the Implementation model to the
Requirements model is TESTING
Mental Model is EVALUATION
Real World is VALIDATION
Interface Model is TRAINING/LEARNING
PROCESSES II
Functional Opacity is created by large differences between the Interface Model and the Mental Model
System Opacity is created by large differences between the Interface Model and the Implementation Model.
Extreme functional opacity or system opacity can lead to cognitive dissonance.
VIRTUALITY
Virtuality is the process whereby the implementation model in the computer replaces the real world model in a system made up in part of people.
Information Systems are usually design on a descriptive basis; however, they often become prescriptive.
Designers need to anticipate prescriptive impacts of their systems.
The design of an interactive system can be the design of a social system
GENERAL SYSTEMS VIEW
Interactive Systems are open, dynamic, and adaptive in the long run
They will be changed by the organizational/social environment and they will influence changes in those environments.
Law of Requisite Variety Applies: The inputs to a system must be as rich/variable as the outputs if one wishes to control the outputs of a system.
FUNDAMENTAL PROBLEM I
Proper Level of Tools / Objects / Data Structure
Too Micro / Low Level
Difficult to work with
More effort and time
Too Macro / High Level
Inflexible
Difficult to Change
FUNDAMENTAL PROBLEM II
Gulf of Execution
Translating goals, intentions to actions
Functional Opacity
Gulf of Evaluation
Translating output display to evaluation
Not System Opacity but related
Excessive trial and error
PLAYING MUSIC PROBLEM
What are the principle alternative ways to make music?
Designing a system any idiot can use and only idiots will use it!
Novice Friendly, Experienced Hostile
Two Separate Dimensions
Beginning users / Experienced users
Non Domain Experts / Domain Experts
COMMON PSYCHOLOGICAL EFFECTS
Fishbowl
Bully
Peephole
Concrete
Clutter (Confusion)
People Angst
Computer Angst
Rorschach Blot
PSYCHOLOGICAL ROLES (Rorschach Blots) OF COMPUTERS
Evaluator, Magician, Helper, Entertainer, Companion, Challenger, Foe, Mentor, Accomplice, Producer, Overseer, Dictator, Priest, Servant, Picky Parent, Unruly Child, God, etc.
Computers can be made to appear to be emotional and/or intelligent.
Providing computer literacy almost an ethical issue
USER RESPONSE TO INADEQUATE SYSTEM
DIS-USE: Turn to other sources
MIS-USE: Using inappropriate ways
PARTIAL USE: Use of wrong subset
DISTANT USE: Use of intermediary
MINIMUM USE: Recipe (IRS type) interaction
NON-USE: Avoiding the system
MODIFICATION OF TASK: Change the task to fit the system (solving wrong problem)
COMPENSATORY ACTIVITY: User has to do more
DIRECT PROGRAMMING: User modifies system/System encourages unique interfaces
Sources of Knowledge and Wisdom I
General Social Sciences Methods
Controlled Experiments
Focus Groups
Interview Methods
Participant Observation
Survey Methods
Longitudinal studies
Field Studies
Sources of Knowledge and Wisdom II
Industrial Engineering
Human Factor studies
Physiological response studies
"time and motion" studies (Method)
Human Monitoring (Method)
Psychology
Human cognitive processes
Human problem solving
Protocol Analysis (Method)
Human psychology and mental ability measurement
Sources of Knowledge and Wisdom III
Management Sciences
Cost/Benefit Analysis (Method)
Efficiency, Effectiveness, Quality, Opportunity, Regret, Productivity
Delphi (Method)
Perceptions and System Monitoring
Measures of user satisfaction, efficiency of use, quality and effectiveness of applications
Management/Operational practices
Performance of Systems/Technology
Sources of Knowledge and Wisdom IV
Sociology
Conceptions of technology
Social and organizational systems
Group processes and behavior
Environmental interaction
Information Science
Data base Technology
Cognitive use of data and information
Information Retrieval
Sources of Knowledge and Wisdom V
Anthropology
Understanding metaphors and culture (Method)
Cultural adaptation
Human learning process
Sources of Knowledge and Wisdom VI
Archaeology
"study of artifacts" (Method)
Tools and toolmaking
Philosophy
"virtual reality"
Design of Social Systems
Virtuality
Sources of Knowledge and Wisdom VII
Computer Science & Information Systems
Designer Wisdom
Expert Systems
Interface Design
Applications
Computer Mediated Communications
Systems Analysis & Requirements Development
Process Reengineering
Enterprise Design
DIMENSIONS OF INTERFACE DESIGN
Subject two
© copyright 1998 Murray Turoff
CRITERIA FOR FACTOR DIMENSIONS
Can it be perceived?
Can it be measured?
Reproducible (repeatable)
Reliable (valid measure)
Orthogonal (Fundamental/Unique)
Can it guide design?
Relate to Interface design choices
Can it evaluate design?
Used to compare different designs
Can it sensitize designers?
Make them better designers
BASIC CATEGORIES OF DIMENSIONS
Foundation Factors
Understanding & Ease of Learning
User Sense of Control
Effectiveness
Psychological & Sociological
Administrative & Management
FOUNDATION FACTORS
Timeliness (Objective)
Responsiveness (Cognitive)
Reliability
Accessibility / Convince
Efficiency / Least Effort
Security
Accuracy
Protection/Bullet-proofing
UNDERSTANDING / EASE OF LEARNING I
Guidance / Informativeness
Conciseness / Brevity
Clarity / Simplicity
Comprehension
Segmentation / Decomposition
UNDERSTANDING / EASE OF LEARNING II
Consistency
Retention
Specificity
Familiarity
Scaffoling
SENSE OF CONTROL I
Leverage / Modifiability
Manipulability
Closure / Confirmation / Notification
Feedback / Sense of causality (Synchronous)
Contextual Visibility
Multi-tasking
Process Control / escape / interrupt / restart
SENSE OF CONTROL II
Forgiveness
Transparency
Flexibility / Cognitive Adaptation
Predictability / Regularity
Tracking / Monitoring
Backtracking / Auditing / History
Forecasting / Anticipating / Alerting
Backup / Undo / Listing
EFFECTIVENESS I
Task Functionality
Generality
Matching
Completeness
Abstraction / Leverage
Adaptability / Manipulability
EFFECTIVENESS II
Integration / Connectivity
Resiliency / Robustness / Richness
Relevance of objects and functions
Precision (minimize non relevant material)
PSYCHOLOGICAL & SOCIOLOGICAL I
Ethical / Honest / Private
Aesthetic / Pleasing / Artful
Interesting / Challenging / Fun
Self Image Enhancement
Peer Relations / Status
Sense of Community
Humanization / Polite
PSYCHOLOGICAL & SOCIOLOGICAL II
Satisfaction
Self, System, Group / Organization, Task / Job
Motivation
Expectations / “Self full filling prophecy”
Perceived Utility
Change & Social Inertia
Feeling of Participation
ADMINISTRATIVE
Training / Documentation
Maintenance of System
Job Enhancement
Human Help & Contact
Organization Relationships
System Evolution / Modifiability
Evaluation / Feedback
Costs and Charging
History & Confidence
EXAMPLES OF CONFLICTS AND TRADEOFFS
Comprehension Segmentation
Consistency Efficiency
Consistency Least Effort
Conciseness Informative
Conciseness Closure
Resiliency Ease of Learning
Task Generality Task Matching
Specificity Familiarity
Leverage Manipulability
COMMON DESIGN FAULTS I
Poor Interactive Sequences
Longer to do than manual (e.g. reading)
No tolerance or respect for human error
No flexible parsing (rigid syntax)
Wrong functionality
Start-stop hassle / Interruption handling
COMMON DESIGN FAULTS II
Integration across functions
Poor documentation and help
Inconsistent Metaphors
Lack of flexibility
Applicability to broad range of tasks
COMMON DESIGN FAULTS III
Multiple approaches to a given task
Multiple ways of invoking tasks
Alternative sequencing of functions
Adapt to different user styles / user types
INTERACTION METHODS I
MENUS: comprehension
COMMANDS: flexibility
LISTS: leverage
FORMS: clarity
DIALOGUE: guidance
WINDOWS: multi-tasking
ICONS: object representation
GUI: special representations
DIRECT MANIPULATION: control
INTERACTION METHODS II
MACROS: leverage
MIMICKING / RECORDING: task matching
ANIMATION: event sequencing
MODELS: abstraction
LANGUAGES: semantic representations
SCRIPTING: leverage, task matching
VIRTUAL REALITIES: understanding
AI AND EXPERT SYSTEMS: efficiency
THE DESIGN PROCESS
Subject Three
© copyright 1998 Murray Turoff
DOING DESIGN
Any type of design component can be the stimulus for an idea
Design can occur in any order through the maze of component relationships
Any incident of a component can be the stimulus for the idea process
Design is a non linear thought process
DESIGN COMPONENTS I
Goals and Objectives
User Tasks & Requirements
System Metaphors
System Artifacts/tools
ISSUES
How to make the computer recipe system more valuable to the user than any of the regular physical systems he or she can use?
How to extend the concept of what is in a recipe to be more than what the user initially conceives it to be?
RECIPE SYSTEM OBJECTIVES I
The user should be able to handle thousands of recipes.
Provide ways of organizing the information that is superior to that provided by books and index card files.
Identify useful functionality that is easier to take advantage of on the computer.
RECIPE SYSTEM OBJECTIVES II
Insure that individuals can learn to use this system very easily and be able to accomplish frequently occurring operations with a minimum of effort.
Provide facilities for the user to be able to learn more about cooking.
USER TASK: search requirements
Meals and meal parts (e.g. breakfast, snack, feast, main course, soup, desert, drinks, etc.)
Ingredients (meat, eggs, fruit, vegetables, etc.)
Nutritional (low fat, vegetarian, high protein, low salt, etc.)
Cultural (Chinese, various holidays, religious, ethnic, national, etc.)
Taste (hot, sour, sweet, bitter, coarse, fine, aromatic, fragrant, etc.)
RECIPE SYSTEM METAPHORS
Recipe, Meal
Cookbook
Card File
Scrap Book
Picture Metaphor
Dining Room, Kitchen, Root Cellar, Pantry, Menu (soup plate, etc.)
RECIPE TOOLS/ARTIFACTS
Nutritional analysis
e.g. calorie counter
Shopping list accumulation
User / Family Preferences Profile
e.g. Filters: likes, dislikes, necessary (low fat, diabetic, low salt)
Glossary, Historical Information
Food and meal layout function
TOOL EXAMPLE: RDA calibration
Function to modify Recommended Daily Allowance (RDA) values for individual (lower salt, etc.) which would change nutritional calculation
Total Fat, Saturated fat, Polyunsaturated fat, Monounsaturated fat, Cholesterol, Sodium, Carbohydrates, Dietary Fiber, Protein, Sugars, Vitamins, etc.
Would require a master ingredient list giving all such information per unit of ingredient.
DESIGN COMPONENTS II
Objects / sub-objects
Object Parts
Representations
Shortest: used to shorten lists
Abstract: provide overview
Content: to fill in total object
RECIPE OBJECT PARTS
"title"
Descriptors for indexing
Auxiliary information: e.g. number of servings, side dishes, drinks
List of "ingredients" and the "amount" of each
Set of cooking "instructions"
User notes, markups
Modification alternatives for special occasions, added by user
RECIPE REPRESENTATIONS
Shortest: Unique title
Abstract: title, index terms, auxiliary data
Content: ingredients, instructions, links
OTHER POTENTIAL OBJECTS
Meals
Key word index
Ingredients index
Marked list
MEAL/MENU Object
Classical Meal Object
Appetizers, Drinks, Breads, Soup, Salad, Cleanser, Main course(s), Vegetables, Starch, Desert, After dinner drinks
User should be able to define meal structure for different types of meals
dinner, lunch, breakfast, brunch, family style, picnic, etc.
Compilation of all ingredients for shopping list
DESIGN COMPONENTS III
Functions on objects
Generic (detail depends on object)
Explicit (self defined)
Strategic choice sets
Reactive choice sets
Controls
GENERIC & EXPLICIT FUNCTIONS
Generic:
Get, Find, Create, Modify, Delete
Explicit Commands for Index
Add, Remove (term), Index (as verb) object, Change (term), Merge (terms), Split (term), Order (index), vote
STRATEGIC MENU/CHOICES
|Object |Number | Actions |
|Recipes |3,455 | Get |
|Meals |238 | Find |
|Lists | | Changes |
| Keys |485 | Create / Add |
| Ingredients |9,989 | Modify /Change |
| Marked |35 | Delete / Remove |
EXAMPLE REACTIVE FUNCTIONS
Acting on some subpart of an object
Modify/add notes
Nutritional Analysis
Find Substitute/Alternative
Add to shopping list
REACTIVE MENU SAMPLE
Scan/List/View
Find
Mark/unmark
Add (index term/instruction/ingredient))
Remove (index term)
Controls: Home (escape), End (finish), Help, Show
EXAMPLES OF CONTROLS
Go (how far), back, forward, elsewhere, open/close/size windows, etc.
Print/File (how much): object, screen, page, list, window, link, etc.
Task control, confirm, quit, help, undo, escape, finish, interrupt, continue, copy, cut, paste, open, close, etc.
One can trace the evolution of interface design through the evolution as to what has become a commonly acceptable control “tool”
DESIGN COMPONENTS IV
Modifiers and Status states
Subsets, tracking
creates meaningful lists
Lateral semantic linkages
Shared processes
List Processing
Searching
MODIFIERS
Create subset/list of objects
Examples
Authored recipes
Incomplete recipes
Sent messages
New messages
LINKAGES I
Meal to recipes
Recipe to meals
Ingredient
as recipe
to details
recipes/meals utilizing it
to substitutions
Number of servings to change program
LINKAGES II
To added Information
cooking method
utensils
nutritional details
Name to similar recipes
Related items: e.g. wine, side dishes
Time to prepare, to cooking
LINKAGE TYPES: Open Ended
Two way usually desirable
Part of (recipe in meal)
List generation (all meals with recipe)
More than one link from object/anchor
Alternative/substitution (of ingredient)
Explanation/Information (about ingredient)
Association/equality (other recipe)
Trigger program (e.g. servings, nutrition)
OBSERVATION ABOUT RECIPE
Natural contextual menu
user knows context
user can be taught context
User can learn easily through trial and error
Ease of learning clear
LIST PROCESSING
Adding and removing: index terms, indexed objects
Dividing, Merging, or Sorting lists
Comparing lists (multiple object entries)
Inverting (all occurrences)
Expanding, Contracting levels
Marking
KEY LIST SAMPLE I
meat
veal
...
beef
sweetbreads
brain
red meat
roasts
...
KEY LIST SAMPLE II
roasts
London Broil
Eye
Rump
...
steak
...
processed
hot dogs
ALTERNATIVE KEY STRUCTURES
Turkey
Holidays
Thanksgiving
Christmas
Leftovers
...
Fowl
Turkey
White meat
Breast
KEY LIST OPTIONS
Meat levels: 9 Items: 455
Use + or ... or "number" to expand or contract
Allow objects in multiple locations
meat.beef.red_meat.steaks.skirt (Shows where it came from)
SEARCH ISSUES I
What specific reactive searches are needed throughout the interface
Initiating the search and being informed of the results with respect to the amount of material found.
SEARCH ISSUES II
Deciding and indicating what is to be retained or discarded in a search result.
Whether to initiate a search of material found or to search the material not retrieved and how to combine this with the prior searches.
DESIGN COMPONENTS V
Object Formats
Screen layouts
Workspace, status areas
Control areas, message areas
OBJECT FORMAT
Rattlesnake Stew
Created: 8/12/91 Modified: 6/5/92
Keys: /meat.snake/exotic/rich/
American.southwest.Indian/
Servings: 2 Preparation Time: 90 min.
Last used: 12/8/92 Times used: 15
Characteristics: High Fat, Stewed
Utensils: Simmering Pot
Remarks: Supplied by John Franklin from my National Guard unit. Tastes very much like rich oily chicken.
INTEGRATION EXTENSIONS
Electronic Shopping
Maintaining inventory
Exchange recipes with other users
Group Meal Planning
Education: Virtual World Recipe System
Being able to actually replicate the cooking process in the virtual world.
Being able to have an instructor join you in that environment.
Being able to cook as a group.
DESIGN COMPONENTS VI
Processes and Closure
Processes and events that take place independent of user.
Interaction process & modes
The states and modes a user can be in.
User Object lists
The lists of objects the user will need or desire to manipulate.
DESIGN COMPONENTS VII
Error conditions
Identification of possible errors that the user could make.
System Messages
Notifications, Closures, Confirmations, and Error Messages.
Necessary Help
Identification of what is unclear or not self evident in the interface.
DESIGN COMPONENTS VIII
For Specialized Systems
CMC: roles, notifications
DATABASES: forms, entity models, reports
COMMUNICATIONS: protocol scripts/templates
HYPERTEXT: anchor, linkage types
Design Levels I
|GENERAL |SPECIFIC |DETAIL |
|Goals |User Task | |
|Metaphors | | |
|Objects |Heading, Abstract |Object parts |
| |Content |Sub-objects |
| | |Formats |
| | |Linkages |
|Functions / Actions |Generic |Explicit, Controls |
| |Links | |
Design Levels 2
|GENERAL |SPECIFIC |DETAIL |
|Strategic Choice Sets |Reactive Choice Sets | |
|Shared Processes |Searching | |
| |List Processing | |
|Interaction States |Screen layouts |Workspace, Status, Control, & System |
| |Error Conditions |message area |
| |Necessary Help | |
PROTOCOL ANALYSIS
Subject Four
© copyright 1998 Murray Turoff
Language is the dress of thought
Samuel Johnson
WHY DO IT?
Objective: To discover the process a person goes through in solving a problem.
Uses: Learning Cognitive Processes, Developing Expert System material, evaluating interfaces.
KEY ASSUMPTION
Cognitive processes that generate verbalization are subset of those that generate behavior
Example: Lisa Learning, by Carroll and Mazur, IEEE Computer November 1986.
CONCERNS ABOUT THE METHOD
Subjects may have incomplete knowledge of their thinking processes; therefore the record may be incomplete.
Subjects may not have an accurate understanding of the processes of which they are aware.
The thinking process may be distorted by the thinking out loud process.
Ambiguity in language may lead to miscommunications.
NECESSARY ATMOSPHERE
Honesty
No evaluation of subject
No pressure for performance
No introduction of bias
No contamination of mental process (e.g. help)
Reciprocity and Respect
STIMULUS RESPONSE METHOD
Different stimulus may produce different mental behaviors
Do you know the capital of Sweden?
Which of the three: Oslo, Stockholm, or Copenhagen is the capital?
Name the capital of Sweden.
How much does it cost to fly and elephant from Thailand to Los Angeles?
How long is the Nile river?
MODES OF PROBING
Talk Aloud, Think Aloud: While information is attended.
Concurrent Probing: While in short term memory.
Retrospective Probing: After completion of the task.
PROTOCOL ANALYSIS CONDITIONS
Subject asked to verbalize what they are thinking
Subject is not being evaluated
Observer must not participate in process
Observer must not aid the subject
Subject providing knowledge of how they solve or a problem (or learn a system)
PROTOCOL ANALYSIS PROCESS I
Present the subject a written explanation of what is taking place.
Explain that this is to evaluate the system and not them
Explain you are there only to observe and can not help them.
Present in writing a task written the user terminology and the way the user would think about it.
PROTOCOL ANALYSIS PROCESS II
Total time should take between thirty minutes to an hour if no major problem encountered.
Observe and record (video, voice tape, PC interactions, notes and coding)
Only interrupt user for further verbalization if it is unclear what they are doing.
PROTOCOL ANALYSIS PROCESS III
Ask the user to describe what he or she is doing outloud.
Ask the user to go through all the terms on the screen and explain what they think they mean
Ask the user to forecast what they think an action will do
Carry out the task on the system
PROTOCOL ANALYSIS PROCESS IV
Give help only if user is at a dead end
Questions you can ask during if necessary
Why do you do/say that?
What is troubling you?
How do you know that ...?
Why do you do it this way?
Save retrospective questions for end of session.
MAJOR LIMITATION
Can not use it on a task that the user is not familiar with.
Requires training on the task first.
TYPICAL WRITTEN INSTRUCTIONS
PLEASE EXPLAIN:
What you think something means.
What you are trying to do.
What confusion or concerns you have.
What you expect to happen next.
What you don’t know the meaning of.
CODING SCHEMA FOR VERBALIZATION I
EXAMPLE I:
Intentions: goals, shall, will, must, have to
Cognitions: current attention situation, define, understand
Planning: If x than y
Evaluation: Yes, No, Damit, Fine
Changing conditions \ view of the problem
CODING SCHEMA FOR VERBALIZATION II
EXAMPLE II:
Surveying given information
Generating new information
Developing a hypothesis
Unsuccessful solutions
Self reference or criticism
Silence
PROTOCOL ANALYSIS OBSERVATIONS I
Verbalization occurs only 30% to 50% of the time.
Subjects cannot verbalize when:
Reading text
Doing intense cognitive activity
Making choices
Subjects have to slow down to verbalize
Subjects will tend to be more careful as a result of verbalization and observation
PROTOCOL ANALYSIS OBSERVATIONS II
Experts on a problem verbalize a lot more than non experts (double)
Experts have more difficulty verbalizing at a very detailed level with respect to the problem with which they are dealing with.
Experts usually spend more time in planning and in the restructuring of the problem.
INCREASING VERBALIZATION
1. Hold back stimulus or encourage slowness.
2. Segment stimulus (subtasks)
3. Interrupt with pre-arranged signal or set point to pause
e.g. when you are ready to indicate an action, first explain what you think everything means on the screen.
OBJECTIVES FOR INTERACTIVE SYSTEMS I
Determine their understanding of terms in the interface
Understand the cause of errors or misinterpretations
Determining missing functionality or user requirements
OBJECTIVES FOR INTERACTIVE SYSTEMS II
Determining reactions to and utility of features
Determine the utility of a metaphor for learning
Determining the utility of help and guidance
PROTOCOL ANALYSIS ADVANTAGES
A lot less effort than other approaches.
Can be done with prototype or mockup before any coding.
Learning how user approaches task
Finding major mistakes in design
Can learn attitude
Rapid feedback from small samples
Also useful for understanding user task
PROTOCOL ANALYSIS REQUIREMENTS
Subjects must be representative
Three subjects for each distinctive type of user and for each different set of tasks
Instructions simple
Observe only
PROTOCOL ANALYSIS QUESTIONS
Can ask/say:
Please explain your choice.
What are you thinking?
What does that term mean?
Should not ask:
Why did you do that?
What does “append” do?
Have physical signal for interrupt
PROTOCOL ANALYSIS: HOW TO I
A one page explanation to the subject
Set of written tasks in user terms
Subject should only spend 40-60 minutes.
Categorization scheme for recording
Tape record their verbalizations
PROTOCOL ANALYSIS HOW TO II
Retrospective questionnaire for end
Retention of major concepts
Perceived utility of features/functionality
Do not try to test everything
At least three subjects on same task
Be specific about user explaining choice they are about to make
UTILITY OF MOCK UP
Ease of understanding (current knowledge)
Ambiguity in terms
Confusion generation
Loss of continuity
Developing on line help
Marketing and acceptance
UTILITY OF WORKING SYSTEM
Ease of learning (new knowledge)
Error impact & Utility of feedback
Ease of exploration
Realistic task execution
Developing final user documentation
Utility of new features (beyond current system)
Wizard of Oz System
Allows simulation of working system
Dummy output data stored
Working choices for moving among screens and windows
More software to do this appearing on the market
BASIC QUESTIONS I
Do the terms used on the screen mean to the subject what the designer thought they would mean?
Do the alternatives presented at that point in the interaction include what the subject wishes to do?
Is the help material or the system messages understandable or relevant to the needs of the user?
BASIC QUESTIONS II
Does the subject have difficulty locating or perceiving things on the particular screen?
Does the subject utilize the sequences of operations that the designer expected to be utilized in accomplishing a given task?
Can the user utilize the interface metaphor for learning the system?
What type of errors is the user making and why?
WHEN TO USE
Protocol analysis should be used
Before system requirements are finalized
After every major change to requirements or interface
Before instillation of system for users
Before introduction to new user population type
INDEXING & SEARCHING
Subject Five
© copyright 1998 Murray Turoff
"When I use a word," Humpty-Dumpty said, "it means
just what I choose it to mean -- neither more nor less."
--Lewis Carroll, Alice in Wonderland
PART OF THE PROBLEM
Bible, Holy Bible, La Sainte Bible, Biblis Sacra, Bible wordo, biblia, Scriptures, Holy Scriptures, The Scriptures, Sacra Scrittur, Saintes, New Testament, Old Testament, Testament, Nouveau Testament, Avrcien, Vetus Testamentum, vetus novum, nuovo testament, gospels, evangelium secundum, mathhewm, gospel of St. Matthew, epistle to the romans, acts of the apostles, proverbs, psalms ecclesistes.
SEARCH MATCHING PROBLEM
IF DEALING WITH COLLABORATIVE INPUTS TO DATA
Air Force Base
air force base
AFB
A.F.B.
A. F. B.
MULTIPLE MEANINGS: AMBIGUITY
What is a “tank”?
weapon
oil storage tank
fish tank
part of toilet unit
May need context to resolve
Use of jargon and codes in groups
INDEXING TYPES
Hierarchical
Network (lateral)
Subject headings
Key words
Syntactic languages
Phrases
Natural Language
HIERARCHICAL
Single location in tree
Precise object (definition of item)
e.g. 1, 1.1, 1.2, 2.1, 2.1.1
e.g. outlines, MDSE. files
Minimum coding size, difficult to do and use
Rigid, difficult to evolve / adapt
NETWORK (LATERAL LINKS)
Single Location in network
Precise relationship (definition of link)
e.g. book index, citation index
e.g. Hypertext (in theory, not practice)
Lack of global view
CITATION INDEX I
Index specific to a time like last year or last five years
Source of data all professional papers published during that time frame (on line version for all years recorded).
Papers published in refereed journals and conferences
CITATION INDEX II
For a paper published at any time in the past and referenced by papers published in that time frame, the results is a list of which papers referenced the earlier papers.
Uses reference list at back of papers to create an “inverted index”
CITATION INDEX PROPERTIES
Most papers cease to be referenced five to ten years after publication
Most important contributions to science are papers reference ten and twenty years after publication
Some fields have significant number of papers never referenced (e.g. math is 30%)
CITATION INDEX UTILITY
Useful for finding recent work in a given field defined by important papers in the field.
Useful for evaluating the contribution of individual scientists over the long term.
Takes 3 to 5 years after publication to begin to assess importance of individual paper.
SUBJECT HEADINGS
Fixed categories, no structure, categorical scale
Precise definitions of bin in which to fit items: Problem: book on “Use of integral equations in Physics”
Open choice as to file objects in multiple headings but not commonly done except through lateral linkages (e.g. cross references)
KEY WORD AND COORDINATE SYSTEMS
Fixed key words
Free key words
Multiple keys for single object
Coordinates for properties
e.g. tall, medium, or short
SYNTACTIC LANGUAGES: Tagged Descriptors
Qualified keys, associated subcategories
Mix of key word with attached hierarchies
e.g. tank.weapon or tank.petroleum
e.g. steak.beef, steak.lamb, steak.veal
(type of meet).(source)
SYNTACTIC LANGUAGES: Faceted Index
Separate dimensions to represent item
Like the many facets of a diamond
softness, style, color, grade
reference: author, title, source
Very common in science and commerce
leather, wine, metal alloys, plastics
Both objective and subjective dimensions
MIXED INDEX EXAMPLE
UNIVERSAL DECIMAL CLASSIFICATION
341.67:623.454.8(094.2)
341.67: Disarmament, limitation and control of weapons of mass destruction
623.45: Ammunition, pyrotechnic devices, weapons of mass destruction
623.454.8: Active rays, atomic nuclear (thermo) weapons
094.2: Historical sources (09), International treaties
Why is nuclear weapons a subcategory of ammunition?
NATURAL LANGUAGE: Phrases
Short phrases, titles
chapter/book titles, paper titles
e.g. KWIC index
Often poor because done by original author
KWIC INDEX
Key Word In Context
Alphabetical listing of all titles permuted word by word
Words like of, the, a, an etc. not used.
Acceptance of Groupware Systems.
Groupware Systems. Acceptance of
Systems. Acceptance of Groupware
NATURAL LANGUAGE: Abstracts, Paragraphs
Long enough to distinguish the single item uniquely from all others in the collection of items
Good when searchers and writers of abstracts are experts in domain
English ambiguity a major problem
INDEXING EFFECTIVENESS I
Relevant Non Relevant
Retrieved A B
Not Retrieved C D
Precision = A / (A+B)
Recall = A / (A+C)
Specificity = D / (B+D)
Total Size Dependent (D)
Search Efficiency = (Recall)x(Specificity)
INDEXING EFFECTIVENESS II
Cannot have both high precision and high recall with qualitative information and even very complex quantitative databases (e.g. all known plastics, metal alloys, etc.)
Indexing methods represent compromise between these two opposing objectives.
INDEXING EFFECTIVENESS III
Too much recall leads to information overload and/or more effort for user
Too much precision leads to loss of relevant material and poor quality result for user
INDEXING EFFECTIVENESS IV
Timeliness (updating and currency)
Accuracy
Completeness (All data in database)
Form/structure of data useful (e.g. summary, raw data)
Depth and Breath (depth easier for novices, breath better for experts)
INDEXING EFFECTIVENESS V
Discrimination & Resolution / Density
Subjective / Objective
Adoption & Acceptance
Historical relevance
Integration (analysis tools, other sources, etc.)
EVALUATION DIMENSIONS I
INDEX TYPE
AMBIGUITY EXPRESSIVE CONCISE
HIERARCHICAL
LOW LOW HIGH
NETWORK
SUBJECTS
FIXED KEYS
FREE KEYS
TAGGED DESCRIPTORS
FACETED INDEXES
PHRASES
NATURAL LANGUAGE
HIGH HIGH LOW
EVALUATION DIMENSIONS II
INDEX TYPE
RETRIEVAL SELECTION ADOPTION
EFFORT EFFORT EFFORT
HIERARCHICAL
LOW HIGH HIGH
NETWORK
SUBJECTS
FIXED KEYS
FREE KEYS LOW
TAGGED DESCRIPTORS
FACETED INDEXES
PHRASES
NATURAL LANGUAGE
HIGH LOW HIGH
IDEAL USE OF INDEX TYPES I
HIERARCHICAL
Macro concepts, well structured area (table of contents)
NETWORK
Micro concepts, structured relationships (hypertext document)
SUBJECTS
Macro concepts, structured objects (professional papers)
IDEAL USE OF INDEX TYPES II
FIXED & FREE KEYS
Micro concepts, unstructured, evolving (book index, files)
Free if area is evolving
Fixed when creators know more than users
IDEAL USE OF INDEX TYPES III
TAGGED DESCRIPTORS & FACETED INDEXES
Micro, structured factors, well understood dimensions (e.g. materials, commodities)
PHRASES
Macro, semi-structured (message titles, paper titles, book titles, etc.)
NATURAL LANGUAGE
Macro, unstructured (abstracts)
ZIPF’S LAW
Log frequency of terms in the English language (Y axis) are linear with log of rank order (X axis).
Product of frequency and rank order raised to fixed power is a statistical constant
Non log plot often called “draw down curve” and used in other fields.
SAMPLE Zipf List I
Word Rank Frequency R x F
the 1 68,315 68,315
of 2 35,716 71,432
and 3 27,856 83,568
to 4 26,760 107,040
a 5 22,744 113,720
in 6 21,108 126,648
that 7 11,188 78,316
is 8 10,978 87,824
was 9 10,499 94,491
it 10 10,010 100,100
Sample Zipf List II
Word Rank Frequency R x F
sir 195 452 88,140
it's 196 452 88,592
why 197 451 88,847
asked 198 448 88,704
give 199 446 88,754
once 200 443 88,600
Sample Zipf list III
Word Rank Frequency R x F
usually 400 239 95,600
tax 500 167 83,500
ideas 800 128 102,400
proved 1,170 88 102,960
sections 2,146 49 105,154
flames 5,070 17 86,190
cultures 7,020 11 77,220
OBSERVATIONS ON LANGUAGE
Order: Easiest to say to hardest to say (amazing collective evolution of language)
Can be derived from Classical Information Theory by minimizing total length of codes to represent message. Done years later after Zipf’s empirical studies.
CONSEQUENCES
Led to Zipf’s Principle of Least Human Effort
Human Behavior and the principle of least human effort: an introduction to human ecology, George Kngsley Zipf, 1949, 1972.
Concepts extended to many other areas of human and biological activity.
SEMANTIC BALANCE I
Speaker’s economy in possessing a vocabulary of one word with m different meanings.
Auditor’s (listener) economy in possessing a vocabulary of m different words with one distinctive meaning.
Conflict in language and cause of “ambiguity.” A collection of m words will have more than m meanings.
SEMANTIC BALANCE II
First 1,000 words (after initial 500) have like 6,000 meanings, meaning they are used much more often than later words. (Ave. = 6)
First 20,000 words have like 40,000 meanings (Ave. = 2)
Suitable packet (unit of words) size can fit harmonic series
Compromise and tradeoffs in human affairs
Zipf RELATIONSHIPS
Form of equation from: Generalized Harmonic Series
r x f ^ (1/p) = C
r = rank f = frequency
1 >= p >=0
for English p ~ 1, but specialized vocabularies differ slightly.
Distribution of star types provides insight into evolution
ZIPF’s STUDIES I
Number of retail establishments vs. population
Diversity (number of different establishments) vc. population
Vocabulary of NY Times vs. ordinary paper (constant a lot larger)
Tool layout for craftsperson (furniture makers, shoe makers, watchmakers, etc.)
ZIPF’s STUDIES II
Frequency of marriage vs. distance in blocks (before cars)
Visitors to national parks vs. distance of home
Size of cities (derived from agricultural model) before last few decade
Number and diversity of establishments vs. size of population
Biomass (frequency x weight) in pond (closed ecological system) constant among species in food chain.
SYSTEMATIC KEY WORD LIST DEVELOPMENT I
Analyze body of literature (significant sample) in application domain
Compare order to normal English and highlight significant differences
Concordance study useful for any writers
High frequency useless: too many matches
SYSTEMATIC KEY WORD LIST DEVELOPMENT II
Low frequency useless: too few matches
Use middle frequency terms different from normal English
Do correlation studies of two, three, four, and five terms together (within same paragraph, sentence) to seek clusters of matches.
PREPARING THE SAMPLE
Remove useless words (“Exclusion list) from initial sample
Illustration of “a” words from such a list:
a about above after afterall afterward again against all almost already also although always am an and another any anyone anything are as at
Remove suffixes: (e.g. age als ant ary) from all words in sample
SEARCH APPROACHES I
Scanning: Broad, top level, term lists
Browsing: Up down, laterally, following interest, iterative, backup and start over
Target Searching: Specific target objective, need structure of data, categorization rules and understanding
SEARCH APPROACHES II
Exploring: Objective is learning the nature of the database and contents.
Wandering: Creative stimulus, random patterns
Both are a significant design challenge
SUBJECTIVE HUMAN INDEXING
Spatial Indexing (e.g. piles of paper)
Event Indexing (e.g. calendars)
Color & Form Indexing (e.g. tabs, marks)
Categorical (e.g. priorities)
SEARCH PROCEDURES I
1 Perceiving a need to find
information.
2 Formulating the request for
information.
3 Selection of the source for the
information.
SEARCH PROCEDURES II
4 Specifying the search strategy.
4.1 Browse the index, provide
numbers
4.2 Expand and contract levels
4.3 Mark multiple choices (or)
4.4 Provide for: or (novice), and, not
conditions, also distance
conditions
SEARCH PROCEDURES III
5 Carrying out the search
(mechanics).
5.1 Show number of hits
dynamically
5.2 Browse the hit list
5.3 Mark, Nest, Backup, Merge,
Restart
SEARCH PROCEDURES IV
6 Evaluating the results
6.1 Update index or list. Determine
relevance
6.2 Saving, printing, viewing,
categorization
6.3 Merging, reordering, clean up,
store strategy
End of Lecture 1-5 Set
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- course title
- policy sample licensed family child care centers
- nebraska state unit on aging senior center manual template
- birthday trip 2006
- university of minnesota
- effective curriculum and instruction for career education
- chemistry 101l kentucky department of education
- sample questions professor isler
- design of interactive systems
- policies and procedures for catering department
Related searches
- types of economic systems quizlet
- importance of information systems pdf
- types of economic systems pdf
- roles of information systems pdf
- types of information systems pdf
- department of retirement systems wa
- design of analog cmos integrated circuits
- examples of information systems pdf
- management of information systems jobs
- management of information systems pdf
- design of analog cmos integrated circuits 2nd
- aashto design of pavement structures