Usability Test for - Cascadia College
Usability Test Results for
Xolax Corporation’s
Corporate Web Site
Data in this report generated by
Ruth Villesis (ruth.villesis@mind-)
Diana Hendricks (diana.hendricks@mind-)
Martha Porter (martha.porter@mind-)
INTRODUCTION
Following is the test plan for conducting usability tests of the corporate web site for Xolax Corporation. The plan covers the following sections:
Purpose
Problem statements
User Profiles
Methodology
Task list
Test environment and equipment requirements
Evaluation measures
Test report contents and presentation
PURPOSE
The XOLAX Company is a software engineering firm located in Austin, Texas. The company’s main product is a suite of Java development tools called XOLAXware. As per the corporate vision, the purpose of the web site is to provide technical support to its current customers, to act as a sales channel for existing and potential customers, and to act as a vehicle for self-promotion for the company and its products.
The XOLAX web site has recently undergone a major facelift: both content and look-and-feel have been recreated from scratch to produce a more modern, professional look. Additionally, vast amounts of new informational content (primarily in the way of online documentation and technical support) have been added to enhance the value of the site to our users.
The purpose of this battery of tests is to determine if the changes that have been made have produced an overall improvement in site usability, as compared to our previously established baselines (1998). We hope to implement changes suggested by this study before the GOOGOLPLEX trade show in Las Vegas in July.
TEST OBJECTIVES
After analyzing the purpose of this web site, we have determined that there are several broad categories of questions that need to be answered about its usability.
1. Can our current users find the technical support information they need on the XOLAX web site?
2. Can potential customers locate product information and ordering instructions?
3. Can existing customers locate upgrade information and place an order for an upgrade?
4. Can press personnel locate news releases and other related information?
5. Can potential employees find job information and apply for a job?
USER PROFILES
Through research performed by the XOLAX marketing department, we have been able to identify several target audiences for the XOLAX web site. They are classified as two groups, a main target group and a secondary target group. Both of these groups may be further subclassed into more groups.
Main Target Groups
Purchasing authorities for large, multinational hi-tech companies are the main target group for this website. This main target group can be separated into two distinct subgroups.
• Potential Customers - People that already make decisions regarding equipment purchases. They are most likely to be highly educated with advanced degrees in engineering and/or business. They may currently do business with one of our competitors via the web or traditional purchasing channels.
• Current Customers– Also have a high degree of education. Our research shows that this group has a high loyalty rate but that it also expects a high degree of quality and service. One of our focuses is on heightening our level of service to them.
Secondary Target Groups
Members of this category are likely to use the XOLAX web site, but are not considered primary users.
• Investors
• Press/ Media
• Future Employees
METHODOLOGY
The usability test will consist of the main performance test designed to gather extensive usability data via direct observation.
The main performance test is composed of the following four sections:
1. Participant greeting and background questionnaire
Each participant will be greeted by the area receptionist and offered a seat and refreshments. The participants will be given a very short questionnaire that gathers basic background information. Participants will be notified that they will be anonymous throughout results reporting, and that they will be assigned a unique ID number, which will be used in place of their name.
2. Orientation
The participant will listen to a short, verbal script that will explain the agenda for the day. All parts of the test will be explained. The participants will be notified of their right to leave the session should they become uncomfortable.
3. Performance test
During this section of the test, the test moderator will ask the participants to perform a series of usability tasks on the XOALR corporate web site. Each task will be introduced by way of a short, scripted instruction from the test moderator.
For each task, the data logger will record the participants’ actions, including time elapsed, number of clicks, and success/failure. The individual tasks are enumerated below.
4. Participant debriefing
After all tasks are complete or the time expires, the test monitor will debrief each participant using a questionnaire. The debriefing will include the following:
9. Participant’s perceptions about usability and aesthetics of the site
10. Participant’s overall comments about his or her performance
The debriefing session serves several functions. It allows the participants to say whatever they like, which is important if tasks are frustrating. It provides important information about each participant’s rationale for performing specific actions, and it allows the collection of subjective preference data about the site.
After the debriefing session, the participants will be thanked for their effort, and invited to review other sites.
TEST ENVIRONMENT AND EQUIPMENT REQUIREMENTS
Our testing lab will be a simple lab setup, including a computer workstation, and three seats: one for the participant, one for the test monitor, and one for the data logger. A diagram of the setup we used is given below.
[pic]
[pic][pic][pic]
Data Logger User Test Moderator
EVALUATION MEASURES
The following evaluation measures will be collected and calculated:
1. The average time to complete each task, and average number of clicks, across all participants.
2. The percentage of participants who finished each task successfully versus those who had errors from which they could not recover.
3. Error classification: to the degree possible, each error will be classified and a source of error indicated. Error classes are as follows:
❑ Observation and Comments -- The test monitor notes when participants have difficulty, when an unusual behavior, or when a cause of error becomes obvious.
❑ Non-critical Errors -- An individual makes a mistake but is able to recover in the allotted time.
❑ Critical Errors -- An individual participant make a mistake and is unable to recover and complete the task on time. The participant may or may not realize a mistake has been made.
4. Participants ranking of usability and aesthetics of the site. (Some questions may be essay-type, rather than rankings.)
TASK LIST
Task List for XOLAX Web Site Usability Test
TASK LIST LEGEND:
MTC = Maximum time to complete
SCC = Successful completion criteria
|TASK NO. |TASK DESCRIPTION |TASK DETAIL |
| |Locate the recent press release about |SCC: Navigate to the URL shown below. |
| |XOLAX’s involvement with a local | |
| |University. | |
| | | |
| | |MTC: 1 Minute |
| |You would like to apply for a software|SCC: Locate requirements on page listed below. |
| |engineer position. What are the job | |
| |requirements for this position? | |
| | | |
| | |MTC: 1 Minute |
| |What is the availability of the FOO++ |SCC: Availability is: shipping in mid-may. Locate information in one of |
| |software module? |the two following pages: |
| | | |
| | | |
| | | |
| | |or |
| | | |
| | | |
| | | |
| | |MTC: 1 Minute |
| |You have a damaged FOO format file. |SCC: Locate technical article #4982734 via Tech Support Search page |
| |You need to use the recover utility. | |
| |Find out how to use it. |MTC: 1 Minute |
| |After you installed Service Release 8 |SCC: Use Tech Support Search page to locate article # 94875334 |
| |for the XOLAXware compiler, your | |
| |computer has begun crashing. Find out |MTC: 2 Minutes |
| |if this is a known bug. | |
| |What is the cost of the QUUX-12 filter|SCC: Price is $350 – find this via one of the following pages:|
| |plug-in module? | |
| | | |
| | | |
| | | |
| | | |
| | |MTC: 2 Minutes |
| |Is there a competitive upgrade path |SCC: Yes, for $399 USD. Find at following page: |
| |from SPORKTECH’s SNOO++ package? | |
| | | |
| | | |
| | |MTC: 2 Minutes |
| |You want to purchase the XOLAXware |SCC: Navigate to the URL below, fill out form, and complete transaction |
| |Professional package. Please use this | |
| |(fake) credit card to place the order.| |
| | | |
| | |MTC: 4 Minutes |
| |Are there any known XOLAX Y2K or other|SCC: Yes, certain products are not year 2038 compliant. Locate article on |
| |related date problems? |page shown below. |
| | | |
| | | |
| | | |
| | |MTC: 1 Minute |
| |What are the system requirements for |SCC: Windows 98 or NT 4.0 SP6 +, 128 MB RAM, 333 MHz Pentium II or better.|
| |the XOLAX Vaporware suite? |Find on page shown below |
| | | |
| | | |
| | | |
| | |MTC: 1 Minute |
| |What is the maximum size for a FOO |SCC: 128 GB. See page listed below. |
| |bytecode file? | |
| | | |
| | | |
| | |MTC: 1 Minute |
| |Please sign up for a free XOLAX |SCC: Locate form at URL below, fill out form and submit. |
| |developer account. | |
| | | |
| | | |
| | |MTC: 3 Minutes |
RESULTS
Performance Data Summary
| |% of participants |% of participants |Mean Time |Mean clicks |
| |performing Correctly|performing |(minutes) | |
|TASKS |(within benchmark) |Incorrectly (within | | |
| | |benchmark) | | |
|1. Locate the recent press release |0 |100 |1.01 |3.57 |
|about XOLAX’s involvement with a | | | | |
|local University. | | | | |
|2. You would like to apply for a |43 |57 |.67 |4.00 |
|software engineer position. What are | | | | |
|the job requirements for this | | | | |
|position? | | | | |
|3. What is the availability of the |86 |14 |.30 |.86 |
|FOO++ software module? | | | | |
|4. You have a damaged FOO format |100 |0 |.16 |2.43 |
|file. You need to use the recover | | | | |
|utility. Find out how to use it. | | | | |
|5. After you installed Service |100 |0 |.30 |3.43 |
|Release 8 for the XOLAXware compiler,| | | | |
|your computer has begun crashing. | | | | |
|Find out if this is a known bug. | | | | |
|6. What is the cost of the QUUX-12 |57 |43 |1.45 |7.43 |
|filter plug-in module? | | | | |
|7. Is there a competitive upgrade |43 |57 |1.39 |5.86 |
|path from SPORKTECH’s SNOO++ package?| | | | |
|8. You want to purchase the XOLAXware|57 |43 |1.56 |9.29 |
|Professional package. Please use this| | | | |
|(fake) credit card to place the | | | | |
|order. | | | | |
|9. Are there any know XOLAX Y2K or |29 |71 |.80 |3.86 |
|other related date problems? | | | | |
|10. What are the system requirements |86 |14 |.24 |2.57 |
|for the XOLAX Vaporware suite? | | | | |
|11. What is the maximum size for a |100 |0 |.18 |2 |
|FOO bytecode file? | | | | |
|Please sign up for a free XOLAX |29 |71 |1.68 |* |
|developer account. | | | | |
Preference Data Summary
| |%SA |%A |%N |%D |%SD |
|Questions | | | | | |
|2. Was the amount of information on the home page adequate? | | | | | |
| |0 |43 |29 |29 |0 |
|3. Was the use of color appropriate? | | | | | |
| |0 |57 |43 |0 |0 |
|4. Was information grouped consistently? | | | | | |
| |0 |29 |14 |57 |0 |
|5. Was the navigation inherently intuitive? | | | | | |
| |0 |14 |14 |57 |14 |
|6. Were colors and navigation consistent through out the site? | | | | | |
| |14 |43 |43 |0 |0 |
|7. Were you able to move around the site without getting lost? | | | | | |
| |0 |29 |43 |29 |0 |
|8. Was there too much information on individual pages? | | | | | |
| |14 |29 |29 |29 |0 |
|9. Was there too little information on individual pages? | | | | | |
| |14 |43 |14 |29 |0 |
|10. Was there adequate cross-referencing of topics and | | | | | |
|information? |0 |29 |29 |29 |14 |
|11. Was more important information highlighted in some way? | | | | | |
| |29 |14 |14 |43 |0 |
|12. Were topic and page headings self-explanatory? | | | | | |
| |14 |43 |14 |29 |0 |
|13. Was it necessary to scroll often to reach desired | | | | | |
|information? |86 |14 |0 |0 |0 |
|14. Was the “Table of Contents” (commonly called a Sitemap) | | | | | |
|helpful? |29 |29 |14 |14 |14 |
|15. Was the terminology understandable throughout the site? | | | | | |
| |0 |71 |29 |0 |0 |
|16. Was there an adequate use of graphics? | | | | | |
| |29 |14 |14 |43 |0 |
|17. Were the graphics clear and sharp? |29 |29 |29 |14 |0 |
|18. Were text and graphics presented in a visually aesthetic | | | | | |
|manner? |29 |43 |29 |0 |0 |
|19. Was there an adequate use of white space? | | | | | |
| |29 |43 |14 |0 |14 |
|20. Overall, were the pages quick to load? | | | | | |
| |43 |56 |0 |0 |0 |
Appendix A
Orientation Script
Hi, my name is ______________. I’ll be working with you today. Let me give you a little background on what we are doing here.
We’re here to test how easy it is to use the ___________ web site, and we’d like your help. We’re looking for potential design flaws and usability problems within our web site.
I am going to ask you to perform several tasks on the web site, and I’d like you to carry out those tasks to the best of your ability. We are testing the site’s usability, not your ability. Try to work in the same manner that you would at work or at home; don’t worry about paying special attention to details, and don’t feel pressured to succeed, since much of what we are looking for is the natural ease of use of this site.
I’d like to assure you that none of the information that we gather here today will ever be released outside of our research team, and your name will never be used in conjunction with your results. For our study we have assigned you an ID number that will be used on all documentation.
Although we don’t anticipate any of the activities today to cause you any discomfort, you should know that it is your right to stop any task that makes you feel uncomfortable or overly stressed out. You may quit the session at any time.
Since we are trying to determine the site’s usability, during this session I will be unable to answer any questions that would help you accomplish a task; you will need to rely only on the resources in the web site itself for task completion. However, you may ask any questions you’d like now before we begin.
After the session is over, there will be a brief period when you can ask any questions you would like, even about tasks that you were unable to complete. We would then appreciate if you would complete an questionnaire about your experience using the site.
Do you have any questions before we begin?
Appendix B
Post Test Questionnaire
Please answer the following questions based on your experience using the web site.
Legend
SA = Strongly Agree; A = Agree; N = Neither; D = Disagree; SD = Strongly Disagree
Task Questions/Issues:
1. Was the language on the Task List you were given, easily understood?
SA A N D SD
General On-Screen Issues:
2. Was the amount of information on the home page adequate?
SA A N D SD
3. Was the use of color appropriate?
SA A N D SD
4. Was information grouped consistently?
SA A N D SD
5. Was the navigation inherently intuitive?
SA A N D SD
6. Were colors and navigation consistent through out the site?
SA A N D SD
7. Were you able to move around the site without getting lost?
SA A N D SD
8. Was there too much information on individual pages?
SA A N D SD
9. Was there too little information on individual pages?
SA A N D SD
10. Was there adequate cross-referencing of topics and information?
SA A N D SD
11. Was more important information highlighted in some way?
SA A N D SD
12. Were topic and page headings self-explanatory?
SA A N D SD
13. Was it necessary to scroll often to reach desired information?
SA A N D SD
14. Was the “Table of Contents” (commonly called a Sitemap) helpful?
SA A N D SD
15. Was the terminology understandable throughout the site?
SA A N D SD
16. Was there an adequate use of graphics?
SA A N D SD
17. Were the graphics clear and sharp?
SA A N D SD
18. Were test and graphics presented in a visually aesthetic manner?
SA A N D SD
19. Was there an adequate use of white space?
SA A N D SD
20. Overall, were the pages quick to load?
SA A N D SD
Overall impression of the site
On a scale of 1 to 10, 1 being the worst, and 10 being the best, how would you rate this site based on what you have seen today? (Circle 1)
1 2 3 4 5 6 7 8 9 10
Please add any comments or suggestions in the space provided that you feel will help us evaluate the usability of the web site. Positive and negative comments will all be appreciated and addressed accordingly.
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- twice during the semester you will be the teacher of record
- 6th grade reading writing argumentative texts unit
- separation agreement
- teks lesson plan unit plan
- sample memorandum of understanding ms word
- texas academy of mathematics and science
- usability test for cascadia college
- august comancheisd
- model family law exam answers
Related searches
- certo drug test for meth
- diagnostic test for myocardial infarction
- how to pass urine test for marijuana
- hair follicle test for methamphetamine
- how to pass urine test for methamphetamine
- pass hair test for methamphetamine
- how to pass urine test for methampheta
- drinks to pass drug test for marijuana
- pass a drug test for marijuana
- english assessment test for college
- aptitude test for college major
- assessment test for college majors