Final Project: - University of Pittsburgh



Appendix

Table of Contents

Group Contributions 2

Section 1: User Study Data 3

Section 2: Heuristic Analyses 13

Section 3: Interface Mockups and Screenshots 22

Section 4: Miscellaneous Materials 24

Finalized XML Example 24

Group Contributions

Jamie Butler

Jamie was the lead programmer and mainly responsible for parsing the XML into classes usable by our applet. Because of this, he also had to standardize the XML format the rest of the group used. Because of his expertise in Java, he also solved several issues relating to installation of the applet on the web.

Paul Thompson

Paul was the lead interface designer and secondary programmer. He worked from initial sketches to implementations in Netbeans, as well as subsequent revisions to the interface as warranted by the heuristic analyses. Once the main applet was running and online for user testing, Paul took our prioritized bug/design flaw list and fixed the major problems.

Bill DePhillips

Bill assisted Paul and Jamie with some programming, but his main duty was creating four “difficult” loop examples, resulting in over 700 lines of XML. His other main duty was in conducting non-user-based testing, consisting of two rounds of heuristic analysis, once for the prototype and again for the testing version. He was also the primary author of this report and the accompanying Powerpoint presentation.

Phil Cox

Phil was responsible for conducting the user testing. He drafted the questionnaire and collected and summarized its results. He was a secondary author of this report and a secondary interface designer.

Section 1: User Study Data

Below are all the user study questionnaires:

Send completed questionnaire to: pec14@pitt.edu

Name: Michael DePhillips

Thank you for completing this questionnaire to evaluate our version of the "For Loop" problette.  This questionnaire should take no more than about 20 minutes to try a number of the problems and record your impressions.  The purpose of this evaluation is not to rate your abilities or knowledge of C, but the ease of use and helpfulness of the problette.

To begin, go to the following website.  A Java applet will appear in the window.  You must have a Java enabled browser.  Please note the approximate time you spend on each problem.

HTML link:



Tasks:

1. Please rate your familiarity with Java programming from 1 to 5, with 5 being expert: 5

2. Rate the clarity of the Code and Explanation interface.

Clarity:     No Clue:  1     2      3       4       5    Clear

3. Step through the problem using the forward and backwards buttons.  If a question appears, determine the correct answer to the problem and enter it in the answer window.  Continue to the next problem.

Time: About 6 minutes

Rate difficulty of task from 1 (easy) to 5 (hard): 4

Comment on any difficulties encountered: The questions were really confusing most of the time, and I have never encountered loops, that are missing a condition… “for(…; k< ; …)” I thought that was a syntax error.

Evaluation Questions:

Please rate the features of this program on a scale of 1 to 5: 1- poor, 2 – fair, 3 – average, 4 – above average, 5 – excellent.

1. Rate the clarity of the code and explaination panel prior to reading the instructions 5

2. Clarity of the use of the program controls for navigating through the tasks: 5

3. Ease of use of the controls for entering/editing your answers: 5

4. Helpfulness of explanation for a specific problem when incorrect answer is entered: 3

5. Layout and presentation of problems: 4

6. Ease of following instructions: 4

7. Benefit in gaining a better understanding of  Java programming: 3

Additional comments: Suggestions I have would be to be more specific on the questions like instead of saying “What is the value of x when the loop BEGINS” since it is misleading and sounds like you are asking the value of the variable when it is first initialized, you should say something like “what is the value of the x variable at the beginning of the next iteration” or something like that. I liked the GUI it was nice and easy to use, but maybe it was just my computer but a chunk on the right side was gone so some of the explanations were cut off. The questions and examples are repetitive in the beginning and you should get more original with the examples, but maybe that’s because I think I was on the easy level. Anyways, good job I’m sure beginners in java could learn a little from the program.

Send completed questionnaire to: pec14@pitt.edu

Name: Bill Noonan

Thank you for completing this questionnaire to evaluate our version of the "For Loop" problette.  This questionnaire should take no more than about 20 minutes to try a number of the problems and record your impressions.  The purpose of this evaluation is not to rate your abilities or knowledge of C, but the ease of use and helpfulness of the problette.

To begin, go to the following website.  A Java applet will appear in the window.  You must have a Java enabled browser.  Please note the approximate time you spend on each problem.

HTML link:



Tasks:

2. Please rate your familiarity with Java programming from 1 to 5, with 5 being expert: 3

3. Rate the clarity of the Code and Explanation interface.

Clarity:     No Clue:  1     2      3       4       5    Clear

4. Step through the problem using the forward and backwards buttons.  If a question appears, determine the correct answer to the problem and enter it in the answer window.  Continue to the next problem.

Time: About 3 minutes

Rate difficulty of task from 1 (easy) to 5 (hard): 4

Comment on any difficulties encountered: I didn’t understand what the question was asking since I lost highlighting in the middle of the question. Reword the questions. The interface itself is good. It was straightforward to see what to do.

Evaluation Questions:

Please rate the features of this program on a scale of 1 to 5: 1- poor, 2 – fair, 3 – average, 4 – above average, 5 – excellent.

1. Rate the clarity of the code and explaination panel prior to reading the instructions 5

 

2. Clarity of the use of the program controls for navigating through the tasks: 5

3. Ease of use of the controls for entering/editing your answers: 5

4. Helpfulness of explanation for a specific problem when incorrect answer is entered: 1 – When I missed it, it didn’t say why or the correct answer

5. Layout and presentation of problems: 4

6. Ease of following instructions: 4

7. Benefit in gaining a better understanding of  Java programming: 3

 

Additional comments: Aside from question wording and highlighting which I typed above, please provide explanations as to the right answer when you miss it since I still don’t get some of them.

Questionnaire to Evaluate Revised

Java programming “for loop” problette

Send completed questionnaire to: pec14@pitt.edu

Name: ____Bonnie Walker________________________

Thank you for completing this questionnaire to evaluate our version of the “For Loop” problette. This questionnaire should take no more than about 20 minutes to try a number of the problems and record your impressions. The purpose of this evaluation is not to rate your abilities or knowledge of C, but the ease of use and helpfulness of the problette.

To begin, go to the following website. A Java applet will appear in the window. You must have a Java enabled browser. Please note the approximate time you spend on each problem.

HTML link:



Tasks:

1. Please rate your familiarity with Java programming from 1 to 5, with 5 being expert: __3__

2. Rate the clarity of the Code and Explanation interface.

Clarity: No Clue: 1 2 3 4 5 Clear

3. Step through the problem using the forward and backwards buttons. If a question appears, determine the correct answer to the problem and enter it in the answer window. Continue to the next problem.

Time: 10 min.______

Rate difficulty of task from 1 (easy) to 5 (hard): ___2_____

Comment on any difficulties encountered: __seems to be a bug at the end b/c the same question keeps getting asked and the number of questions keeps increasing and the program never ends. And the screen cuts off at the right so you can’t read everything. _______________________

Evaluation Questions:

Please rate the features of this program on a scale of 1 to 5: 1- poor, 2 – fair, 3 – average, 4 – above average, 5 – excellent.

1. Rate the clarity of the code and explaination panel prior to reading the instructions _4___

2. Clarity of the use of the program controls for navigating through the tasks: ___5__

3. Ease of use of the controls for entering/editing your answers: ____5_____

4. Helpfulness of explanation for a specific problem when incorrect answer is entered: ____4________

5. Layout and presentation of problems: ______3_______

6. Ease of following instructions: _____________4____

7. Benefit in gaining a better understanding of Java programming: ____4___

Additional comments: ___________________________________________

____________________________________________________________

____________________________________________________________

____________________________________________________________

Thank You!

Questionnaire to Evaluate Revised

Java programming “for loop” problette

Send completed questionnaire to: pec14@pitt.edu

Name: ___Luis Torrefranca____

Thank you for completing this questionnaire to evaluate our version of the “For Loop” problette. This questionnaire should take no more than about 20 minutes to try a number of the problems and record your impressions. The purpose of this evaluation is not to rate your abilities or knowledge of C, but the ease of use and helpfulness of the problette.

To begin, go to the following website. A Java applet will appear in the window. You must have a Java enabled browser. Please note the approximate time you spend on each problem.

HTML link:



Tasks:

1. Please rate your familiarity with Java programming from 1 to 5, with 5 being expert: _3_

2. Rate the clarity of the Code and Explanation interface.

Clarity: No Clue: 1 2 3 4 5 Clear

3. Step through the problem using the forward and backwards buttons. If a question appears, determine the correct answer to the problem and enter it in the answer window. Continue to the next problem.

Time: _7:00_

Rate difficulty of task from 1 (easy) to 5 (hard): __1____

Comment on any difficulties encountered: Would lose track of where I was in the sample execution.

Evaluation Questions:

Please rate the features of this program on a scale of 1 to 5: 1- poor, 2 – fair, 3 – average, 4 – above average, 5 – excellent.

1. Rate the clarity of the code and explaination panel prior to reading the instructions _4__

2. Clarity of the use of the program controls for navigating through the tasks: __4__

3. Ease of use of the controls for entering/editing your answers: _____5___

4. Helpfulness of explanation for a specific problem when incorrect answer is entered: _______3____

5. Layout and presentation of problems: _____3_______

6. Ease of following instructions: ______4__________

7. Benefit in gaining a better understanding of Java programming: ____4__

Additional comments: ___________________________________________

____________________________________________________________

____________________________________________________________

____________________________________________________________

Thank You!

Questionnaire to Evaluate Revised

Java programming “for loop” problette

Send completed questionnaire to: pec14@pitt.edu

Name: ________Lorrie Lucht____________________

Thank you for completing this questionnaire to evaluate our version of the “For Loop” problette. This questionnaire should take no more than about 20 minutes to try a number of the problems and record your impressions. The purpose of this evaluation is not to rate your abilities or knowledge of C, but the ease of use and helpfulness of the problette.

To begin, go to the following website. A Java applet will appear in the window. You must have a Java enabled browser. Please note the approximate time you spend on each problem.

HTML link:



Tasks:

1. Please rate your familiarity with Java programming from 1 to 5, with 5 being expert: __1__

2. Rate the clarity of the Code and Explanation interface.

Clarity: No Clue: 1 2 3 4 5 Clear

3. Step through the problem using the forward and backwards buttons. If a question appears, determine the correct answer to the problem and enter it in the answer window. Continue to the next problem.

Time: ____1_

Rate difficulty of task from 1 (easy) to 5 (hard): ______1__

Comment on any difficulties encountered: _________________________

Evaluation Questions:

Please rate the features of this program on a scale of 1 to 5: 1- poor, 2 – fair, 3 – average, 4 – above average, 5 – excellent.

1. Rate the clarity of the code and explaination panel prior to reading the instructions ___4_

2. Clarity of the use of the program controls for navigating through the tasks: ___3_

3. Ease of use of the controls for entering/editing your answers: ____4_____

4. Helpfulness of explanation for a specific problem when incorrect answer is entered: _____4_______

5. Layout and presentation of problems: _______5______

6. Ease of following instructions: ________4_________

7. Benefit in gaining a better understanding of Java programming: __4_____

Additional comments: ____1. explanation box cut off on the right side couldn’t see parts of the words.____2. question 9 missing condition values

3. program doesn’t tell you that you have completed all the problems when you finish

____________________________________________________________

Thank You!

Section 2: Heuristic Analyses

Heuristic Analysis for Prototype Version

|HE1 – Prototype Version |

|Name |

|Questions Lack Feedback |

|Evidence |

|Heuristic: Provide Feedback |

|Users do not know if they answered the questions correctly. |

|Explanation |

|Although the focus of the applet is on viewing the loop execution and learning, the questions are a key part of a user's sense of |

|progress. By providing feedback such as number of questions answered correctly, the user will gain feedback as to their level of |

|knowledge. |

|Severity |

|High – The user may become annoyed that no record is kept and may assume the questions are not important and may skip answering |

|them, lowering the educational value of the applet. |

|Possible Solution |

|On the main page, an ongoing total of questions asked and questions answered correctly should be provided. |

|Relationships |

|… |

|HE2 – Prototype Version |

|Name |

|How to Begin Is Unclear |

|Evidence |

|Heuristic: Simple and Natural Dialog |

|Users may not think to look under the File menu for the Start/Restart command. |

|Explanation |

|Since the applet can only support one file, it makes more sense for the applet to automatically load the file on startup rather |

|than wait for the user to do it manually. |

|Severity |

|High – The user may become frustrated and may believe the applet does not work. It may take a long time for them to discover how |

|to start the applet, especially since it is no mentioned in any instructions. |

|Possible Solution |

|The applet should load the initial example automatically. |

|Relationships |

|… |

|HE3 – Prototype Version |

|Name |

|Question Lack Line Highlighting |

|Evidence |

|Heuristic: Minimize User Memory Load |

|The user must remember which line the code was at in order to answer some questions. |

|Explanation |

|When a question appears, the code highlighting is removed, which means the user must recall which line execution was at in order to|

|answer the question. This is a difficult task for some users, especially since it is not clear when a question may appear. They |

|may click through quickly and suddenly become lost when the question removes important information that is not in their working |

|memory. |

|Severity |

|High – The user may miss questions they could answer and become frustrated with the applet and stop using it. |

|Possible Solution |

|Code highlighting should be added to the question area. |

|Relationships |

|… |

|HE4 – Prototype Version |

|Name |

|Forward & Next Buttons Confusable |

|Evidence |

|Heuristic: Simple and Natural Dialog |

|The user sees two confusable buttons, one “forward” and one “next” very close on the screen yet they perform very different |

|functions. |

|Explanation |

|The button to step forward a line of code is right above the button to move to the next problem. These buttons should be separated|

|using more space so that they are not so close. Also, the “Next Problem” button should be made inactive until the user reaches the|

|end of the problem since this applet is meant to be a sequential exploration, not skipping through problems. Also, since both are |

|labeled with short words, at a glance it is impossible to tell them apart with moving the eye and reading the heading they are |

|under. They should changed to appear more different at a glance. |

|Severity |

|High – The user may accidentally click “Next Problem” when they just meant to move forward one line of code and lose their place. |

|Possible Solution |

|Separate the buttons, change the wording, possibly using icons for the code buttons so that they stand out, and possible inactivate|

|problem control while in the middle of a problem. |

|Relationships |

|HE5 |

|HE5 – Prototype Version |

|Name |

|Code Control buttons too far from Code |

|Evidence |

|Heuristic: Simple and Natural Dialog |

|The buttons to control the code are far from the code. Thus the user's eye must travel back and forth over large distances while |

|iterating. |

|Explanation |

|The main activity in the applet is iterating through code. Thus, it is unnatural to click on the next button and move the eye |

|relatively far to see the change in the code. The user would expect the code control buttons to be either next to or beneath the |

|code, or even within in, but not diagonal to it. |

|Severity |

|Medium – While not a major difficulty, by the end of all the problems, the user may suffer greater eye fatigue than with a better |

|design. |

|Possible Solution |

|The code control buttons should be moved to as close to the code as possible. |

|Relationships |

|H4 |

Testing Version Heuristic Analysis

|HE1 – Testing Version |

|Name |

|Unneeded Scroll Bars |

|Evidence |

|Heuristic: Simple and Natural Dialog |

|Simplicity would dictate that there are no unnecessary widgets, and the scrollbars are unnecessary. |

|[pic] |

|Explanation |

|There are two scrollbars for output windows that do not require them. |

|Severity |

|Low – The user is not terribly distracted but they make the interface appear more complicated than it needs to. |

|Possible Solution |

|The scroll bars should be removed. |

| |

|HE2 – Testing Version |

|Name |

|Confusing Word Wrap |

|Evidence |

|Heuristic: Minimize User Memory Load |

|Because of the non-conventional word splitting the user must remember a word as his or her eye moves to the next line, causing |

|difficulty in reading. |

|[pic] |

|Explanation |

|Conventional word splitting in newspapers or books occurs only on a syllable and utilizes a “-“ symbol to inform the reader of the |

|wrap. This explanation box shown above cuts off a word with no “-“ symbol. Also, in such a small window with short explanations, |

|it is not necessary to even split words. It could simply not break apart words and wrap them as a whole. |

|Severity |

|Medium – Since it is an educational applet and the user is trying to learn from the explanations, the potential slowdown and |

|difficulty in reading caused by this problem could result in a user not understanding an example/ |

|Possible Solution |

|Words should not be split across lines. |

| |

|HE3 – Testing Version |

|Name |

|Questions Missing Code Highlighting |

|Evidence |

|Heuristic: Minimize User Memory Load |

|When a question is asked the code highlighting vanishes, disorienting the user. |

|[pic] |

|Explanation |

|Many questions are related to a specific line of code but when a question appears the code highlighting present during the rest of |

|the iterations vanishes, making it impossible to tell for some questions what exactly is being asked. |

|Severity |

|High – The user may miss questions because of a lack of context and become frustrated. |

|Possible Solution |

|The highlighting should remain during questions. |

| |

|HE4 – Testing Version |

|Name |

|Wrong Answers Do Not Show Right Answers |

|Evidence |

|Heuristic: Provide Feedback |

|While the applet tells a user whether or not they answered correctly, it does not show the right answer when they miss a question. |

|[pic] |

|Explanation |

|The purpose of the applet is learning and while some of the answers are provided through explanations or variable displays, the |

|user is not directly told the correct answer if they miss a question. This makes them unsure if they are typing the answers in |

|incorrectly or some other reason since there is no way to tell how it is processing the correctness of an answer. |

|Severity |

|Medium – The user may not learn as easily from the applet because they cannot learn from their mistakes without poring over the |

|explanation and variable panels in detail, possibly going a few steps backward to check. |

|Possible Solution |

|Display the correct answer when the user submits a wrong answer. |

| |

Section 3: Interface Mockups and Screenshots

A Chronology of Development

[pic]

Early Prototype Mockup in Photoshop

Note:

Several early designs are sketches and are included in the paper copy only. Those viewing this electronically cannot see these artifacts.

[pic]

Section 4: Miscellaneous Materials

Finalized XML Example

1. int matrix[3][3];

2. for(int i=0 ; i < 3 ; i++) {

3. for(int j=0 ; j < 3; j++) {

4. matrix[i][j] = i + j;

5. System.out.print(matrix[i][j]);

6. }

7. }

1

Values in matrix undefined

2D Array of integers created but not initialized

. . .

3

i=1,j=0

Initialization statement of j loop is executed. j initialized to 0

012

What is the value of j before this statement?

undefined

difficult

tracing

. . .

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download