A Survey on Educational Methods Using “CollabTest”, a Web ...

SELECTED TOPICS in EDUCATION and EDUCATIONAL TECHNOLOGY

A Survey on Educational Methods Using "CollabTest", a Web-based Learning System Enabling Students to Create Quizzes Collaboratively

MASANORI TAKAGI1), TETSUYA KANEKO2), MASAMITSU MOCHIZUKI3), JUN SASAKI1), and YOSHIMI TESHIGAWARA4)

1) Faculty of Software and Information Science, Iwate Prefectural University 152-52, Sugo, Takizawa, Iwate

2) Center for Excellence in Teaching and Learning, 3) Faculty of Business Administration, 4) Faculty of Engineering, Soka University, 1-236, Tangi, Hachioji, Tokyo JAPAN

{takagi-m, jsasaki}@iwate-pu.ac.jp, {ktetsuya, mochi, teshiga}@soka.ac.jp

Abstract: In 2002, we developed a web-based learning system named "CollabTest" that enables learners to acquire knowledge by creating quizzes and sharing them with their peers. To date, we have employed this system in 158 classes covering subjects such as computer networks, statistics, and Western history. However, educational models and methods to improve the educational effectiveness of using such a system are not clear. For this reason, the results of using CollabTest differ widely among teachers. In this paper, we report methods for effective utilization of the system by using feedback obtained from teachers via questionnaires and interviews.

Key-Words: Problem Posing, Quiz Creation, Peer Review, Online Test, Educational Method

1 Introduction

Recently, e-learning systems which enable learners to independently create quizzes have been actively studied [1][2][3][4][5]. Moreover, some systems have been used at elementary schools and universities. We previously developed a web-based learning system named "CollabTest" that enables learners to acquire knowledge by creating quizzes and sharing them with their peers [6][7][8]. A noteworthy feature of our system is the collaborative environment for quiz creation. The environment provides a peer-review function that shares a learner's quiz with their group and allows group members to interactively provide feedback. The basic procedure for using CollabTest is (1) quiz creation, (2) peer review, and (3) online testing. We have developed the CollabTest system to enable students to perform these tasks in a web-based environment. Moreover, we have employed this system continually since 2002 at educational institutions of various levels, including a university, high school, elementary school, and vocational training school. In 2003, we developed a points system in which students earn points by performing tasks such as submitting a quiz or posting a comment. Our study project was selected for the Support Program for Contemporary Educational Needs in the 2007 academic year. The Support Program was initiated by the Ministry of Education, Culture, Sports, Science

and Technology to encourage the revitalization of higher education by providing financial support to exceptional educational projects proposed by Japanese institutions. Moreover, an English version of CollabTest was also developed. At present, introduction of our system to Soka University of America and some Chinese Universities is under consideration.

CollabTest has been used in a total of 158 courses over 8 years, and 7856 learners and 53 teachers have used the system. In addition, learners have created 21165 quizzes and posted 50890 comments. From these practical studies, we have demonstrated that CollabTest has the potential to improve study time effectively and students who have actively used it have improved their test scores [9]. However, educational models or methods to improve the educational effects using CollabTest are still unclear. For this reason, the outcomes of using the system differ widely among teachers. For example, because the system provides many functions, it is difficult for teachers to determine which functions they should use and how to use them in order to achieve their course objectives. Furthermore, related studies clarify neither models nor methods.

The purpose of this study is to determine the models and methods for using CollabTest effectively. In addition, we intend to develop a guideline for

ISSN: 1792-5061

318

ISBN: 978-960-474-232-5

SELECTED TOPICS in EDUCATION and EDUCATIONAL TECHNOLOGY

teachers that describes the procedures and precautions for promoting learning through quiz creation. In addition, to achieve our objectives, we issued questionnaires to and conducted interviews with teachers to analyze how CollabTest has been used in their classes.

2 CollabTest

2.1 System Outline

CollabTest is a web-based application developed in JSP and HTML, and uses Java Servlets. Therefore, it is accessible through a standard web browser anytime anywhere as long as an Internet connection is present. In addition, because CollabTest uses Lightweight Directory Access Protocol authentication, it can communicate with other types of authentication systems managed by other educational institutions. CollabTest also enables authentication via a single user account from the Soka University portal, an exclusive service for students of Soka University. Users who are not affiliated with Soka University can be certified by using an authentication function of the system. Furthermore, communication between a web browser and a web server is secured by SSL encryption.

CollabTest supports import and export of Comma Separated Value (CSV) files. This makes it easy for an

administrator to import student data and teachers can export quizzes created in their courses. In addition, there is a function to synchronize data with information systems of other educational institutions. CollabTest uses MySQL as a database server and character encoding is standardized to UTF-8. Therefore, CollabTest can record content in both Western and Asian languages, including English, Japanese, Chinese, and Korean.

2.2 CollabTest Learning Procedure

Figure 1 shows the learning procedure for creating, reviewing, and taking quizzes as performed by learners in a particular class: I. Create quizzes with explanation of quiz content. II. Review the quizzes collaboratively in a group. III. Submit quizzes to their teacher. IV. Take student- or teacher-created quizzes to test

their comprehension. This procedure is conducted during or after class. In many courses that have used CollabTest, explanation on how to use the system was provided only during the first class, and thereafter procedures were conducted as homework. Before learners can use CollabTest in a particular course, teachers need to divide their students into groups and enroll them into the system. In addition, teachers have to create and register category items for quiz creation. Such category items correspond to the

Step 1

Quiz Creation Creating quizzes and

sharing them with group.

Step 2 Peer Review Assessing quizzes interactively

and posting comments.

Student

Improving own quizzes based on comments.

Collaborative quiz creation environment

Quiz

CollabTest

Posting Comments

Teacher

Step 3 Quiz Submission

Taking online tests.

Step 4 Evaluating quizzes and

creating online tests by

Online Test selecting them.

Fig. 1. CollabTest Learning Procedure Diagram

ISSN: 1792-5061

319

ISBN: 978-960-474-232-5

SELECTED TOPICS in EDUCATION and EDUCATIONAL TECHNOLOGY

scope and learning units of quizzes and require registration for each course. Learners consult various items when they create a quiz. When learners register a quiz, they must select a category item registered by their teacher as all quizzes are managed using category items.

After learners start using the system, teachers need to assess the quizzes created by their students, post comments, and create tests. In past practical studies of the system, students who were a teaching assistant (TA) or student assistant (SA) have supported the above-mentioned tasks.

No

Function

Table 1. Main Functions of CollabTest Explanation

Enables administrators, teachers, TAs, and SAs to register, edit, and delete category items. When a new category item is registered, 5 quiz-creation points

are made available. Students must select a category item when they register a

1

Category Management

quiz. Then, the first student who chooses to submit a new quiz in this category will be awarded the 5 points. Next, quiz-creation points for the selected

category will be automatically decreased one by one until a preset minimum

value is reached. The Category Management Function also enables setting

maximum and minimum point values.

2

Group Management

Enables administrators, teachers, TAs, and SAs to register, edit, and delete groups.

Enables all users to register, edit, and delete quizzes. Multiple-choice, fill-in-the-blank, and written questions are options that can be registered. Quiz 3 Quiz Creation information consists of references, keywords, questions, answer options (multiple-choice and fill-in-the-blank), and explanations. An image file and an MS Word, Excel, or similar file can be attached to the question and explanation.

Enables learners to share their quizzes with their group and to post comments

interactively for the purpose of collaborative review. It also provides a function

4

Review

which enables learners to assess quizzes based on review items which we developed. Moreover, it provides a navigation function for peer review which

can guide learners toward an appropriate review process for improving quizzes

[7].

5

Quiz Access Control

Enables administrators, teachers, TAs, and SAs to control access to quizzes by specifying source and destination groups.

Enables administrators, teachers, TAs, and SAs to set the following: number of

Number Settings and quizzes (0?10 or no limit) which students can submit, submission requirements,

6

Submission number of students who posted comments, number of comments posted by

Requirements peers, and number of students who reviewed the quiz. Only quizzes which meet

the submission requirements can be submitted.

Enables students to engage in free discussion. First, a teacher provides a theme. 7 Free Discussion Next, learners prepare a discussion topic and begin discussion. In addition,

learners can rate each posted opinion.

Enables learners to earn points by performing tasks such as submitting a quiz or

8

Point System posting a comment. The point system also allows learners to compete against

others (individuals or groups) via an individual or group point-ranking function.

Enables administrators, teachers, TAs, and SAs to register, edit, and delete

tests. Teachers can use student- or teacher-created quizzes as test items. In

9

Online Test

addition, teachers can also set properties, such as online test answer points, access control, and frequency that students can answer a test, for each paper.

Learners can then answer tests activated by their teacher. If a learner scores

100%, they are awarded an online test answer point.

10

Exchange Learning

Enables student-created quizzes and online tests to be shared between similar courses or identical courses in a different time slot [8].

ISSN: 1792-5061

320

ISBN: 978-960-474-232-5

SELECTED TOPICS in EDUCATION and EDUCATIONAL TECHNOLOGY

2.3 Main Functions

We have repeatedly expanded various functions of the CollabTest system to accommodate user requests. The main functions of CollabTest are shown in Table 1. Teachers use our system by selecting appropriate functions to achieve their course objectives.

Figure 2 shows an item format selection page. Our system enables registration of multiple-choice, fill-in-the-blank, and written questions. In addition, supporting media, such as an image file, can be attached to the question or explanation. This feature makes our system adaptable to various courses, such as certification courses like the TOEIC examination which use a multiple-choice format, or a chemistry course which needs to display pictures and charts.

Figure 3 shows the quiz review page using review items. CollabTest requires students to answer a quiz before they can review it. The correct answer and explanation are displayed only when the answer is submitted. After learners have read the explanation, they can enter a comment or assess the quiz using a navigation function for peer-review.

2.4 Support Staff

The CollabTest system has been managed by the ICT education and promotion division of Soka University since the 2007 academic year. We have since opened a help desk where one part-time staff member and some TAs and SAs are always on hand. These support staff deal with system troubles and explain system usage to each class. Moreover, they supported teacher tasks such as posting comments and reviewing quizzes.

3 Survey and Consideration

We investigated the effective methods for utilization of the CollabTest system by questionnaires and interviews. The investigation was conducted over 3 years from the semester beginning in September 2007 to the semester beginning in April 2010.

3.1 Methods for Effective Use of the CollabTest System

We asked Teachers: "What did you do to use the CollabTest system effectively?" Then we analyzed responses from teachers who had used it effectively. As shown in Table 2, responses were divided into two groups: the first group focused on methods for making students create quizzes more effectively and the second group focused on methods that promoted

Multiple-choice question

Fill-in-the-blank question

Written question Fig. 2. Item Format Selection Page.

Question Evaluation items

Choices

Explanation

Fig. 3. Review Page for a Quiz. learning in the overall learning process displayed in Fig. 1. From these results, we identified that clarifying the quiz theme and the number of quizzes which students had to create, and giving learners time to create quizzes during a class are effective methods when teachers ask students to create quizzes.

ISSN: 1792-5061

321

ISBN: 978-960-474-232-5

SELECTED TOPICS in EDUCATION and EDUCATIONAL TECHNOLOGY

Table 2. Results of Questionnaire 1: What did you do to use the CollabTest system effectively?

Methods for making students create quizzes more effectively

I gave my students time to create quizzes during classes because some students did not have their own PC. (Assist. Prof., Faculty of Engineering; Prof., Faculty of Business Administration)

I gave my students information and hints on creating quizzes. (Prof., Faculty of Education)

I limited the scope of a quiz theme. (Assoc. Prof., Faculty of Letters)

I used the system in a step-by-step approach and placed more emphasis on quality than quantity, which enabled my students to create a better quiz as a group. (Assist. Prof., Faculty of Letters)

I showed my students the number of quizzes which they had to create to promote quiz creation. (Assist. Prof., Faculty of Engineering)

I promoted self-learning by placing more emphasis on quantity than quality. (Assoc. Prof., Faculty of Business Administration)

Methods for promoting learning in overall learning process

I made my students take tests to confirm their comprehension levels in each class. As test items, I used not only quizzes created by my students but also quizzes created in my past courses. In addition, some student-created quizzes created in the present course were used in the final examination. (Prof., Faculty of Engineering)

I adjusted the amount of overall homework so it did not increase the students' workloads. (Assist. Prof., Faculty of Letters)

I set deadlines for submitting quizzes and for answering tests. (Assoc. Prof., Faculty of Economics; Assoc. Prof., Correspondence Education Division; Prof., Faculty of Engineering).

I evaluated points awarded by the system to count for 25% of a student's grade. This resulted in many quizzes and comments being registered. (Assoc. Prof., Faculty of Business Administration)

I gave my students opportunity to engage in discussion with their peers. In addition, I reorganized the group once every 5 classes. (Assist. Prof., Faculty of Engineering)

Moreover, the results show that setting deadlines for submitting or answering tests is important to ensure that these tasks are done. In addition, we confirmed that reorganizing the group periodically is an effective strategy. However, group reorganization is a

time-consuming process. Therefore, we will develop a function that automatically reorganizes a group.

3.2 Teaching Method Using CollabTest

We interviewed some teachers to investigate the effective educational methods of the CollabTest system. From the results, we confirmed the following 3 educational methods.

3.2.1 Quiz Creation and Peer Review during Class In the course "Information Processing-e", which focused on the beginner level Systems Administrator License and was taught at Soka University, a teacher gave his students time to create quizzes during each class. Half of the students of each group were made to create quizzes while the others posted comments and reviewed quizzes created by their peers. In doing so, the teacher expected not only to increase the number of comments for peer reviews and opportunities for face-to-face communication, but also decrease the students' workload. Six quizzes per student were created, while 11 comments per student were posted during the course.

In the course "Information System Theory and Engineering", taught at Iwate Prefecture University, a teacher effectively conducted tests and quiz creation during classes. He conducted a test during the first 30 minutes of the class to assess the comprehension level of the contents of previous lessons. The tests consisted of 10 quizzes, which were produced by correcting and integrating quizzes created in previous classes and then used as test items. First, students answered the test on paper. Next, each student marked another student's test. Then the teacher explained each quiz to the class. Thereafter, he taught new contents for 30 minutes and finally encouraged learning by asking students to create a quiz on the new course content.

3.2.2 Cooperative Reuse in Two or More Courses In the course "Computer Network Theory", taught at Soka University, a teacher opened some tests on the system after each class. He used quizzes created in not only the present course but also past courses as test items.

In the "TOEIC Intermediate Course", taught at Soka Women's College, students did not create any quizzes. However, the teacher conducted online tests during each class using quizzes created in the similar course "Seminar A", in which English language

ISSN: 1792-5061

322

ISBN: 978-960-474-232-5

SELECTED TOPICS in EDUCATION and EDUCATIONAL TECHNOLOGY

instruction was the objective. In "Seminar A", 16 students created 770 quizzes throughout the first semester. This was the most active course in which CollabTest has ever been used. The teacher provided thorough instructions for describing the explanations in detail and evaluated all quizzes. As a result, the quizzes received favorable comments such as "the explanations were written in more detail than any commercial reference books" and "the explanations were easy to understand."

3.2.3 Integration of CollabTest Learning and Classroom Learning A teacher used the CollabTest system in a classroom which had no PC. In the course "Teaching Method", taught at Soka University, he effectively integrated learning in CollabTest with learning in the classroom. He also adopted face-to-face workgroups in each class. As one of the tasks, the teacher made students create quizzes. The flow of this course is shown below. I. Students create more than 3 quizzes each. II. Each student answers all quizzes created by their

group members, then posts more than 5 comments for each quiz. III. Students select 2 quizzes from their group and submit them to their teacher. IV. The teacher opens all quizzes submitted by students as online tests in the course. V. The teacher selects appropriate quizzes from those submitted and conducts class tests. In the above-mentioned tests, the teacher placed each quiz on a MS PowerPoint slide and displayed it to the students with a projector. Students responded to the quizzes using a clicker, which is an audience response method [10]. This method enabled the teacher and his students to display test results on the slide in real time. The test consisted of 10 quizzes and students took the test twice; first, they answered the test on their own, then they answered it as a group. Moreover, the teacher evaluated his students in his own unique way without using the point system of CollabTest. Students could obtain 2 points when they completed each of the tasks of quiz creation, comment posting, and quiz submission, for a total score of 6 points. In addition, they could obtain a further 4 points when they scored 100% in individual tests. However, 1 point was deducted for every incorrect answer. In the group tests, 2 points were awarded for scoring 100% and the same amount were deducted for an incorrect answer.

3.3 Other Methods for Using CollabTest

We developed the CollabTest system as an environment for students to create quizzes in order to intensify their learning. However, we considered that our system is also capable of providing an environment for teachers or TAs to create quizzes. At the Kyushu Institute of Technology, our system has been used by a teacher and TA to create and store quizzes which are used in the "Information Processing" course. The teacher and the TA were both registered as student users, then they created quizzes and reviewed them interactively.

4 Conclusion

In this study, we analyzed the effective uses and teaching methods of the CollabTest system from the results of questionnaires and interviews. In the future, we will develop new functions to support tasks which are not yet performed on the system. In particular, we plan to develop functions that do not limit teaching methods in any way. Our final goal is for the system to run using only the minimum functions which will be selected automatically when a teacher selects course objectives, utilization purpose, and expected outcomes. Next, CollabTest will automatically display a guideline showing ways of using the loaded functions. Although there were many teachers who included points attained from CollabTest exercises in students' grades, the validity of these points is not yet established. Therefore, we will also consider ways of evaluating learning activities performed in CollabTest.

Acknowledgment

The work reported in the paper was supported in part by Grant-in-Aid for Scientific Research (B) (No. 21300315) from the Ministry of Education, Culture, Sports, Science and Technology of Japan.

References: [1] M. Barak and S. Rafaeli, On-line question-posing

and peer-assessment as means for web-based knowledge sharing in learning, International Journal of Human-Computer Studies, Vol.61, No.1, 2004, pp.84-103. [2] Fu-Yun Yu, Yu-Hsin Liu and Tak-Wai Chuan, A web-based learning system for question posing and peer assessment, Innovations in Education and Teaching International, Vol.42, No.4, 2005, pp.337-348.

ISSN: 1792-5061

323

ISBN: 978-960-474-232-5

SELECTED TOPICS in EDUCATION and EDUCATIONAL TECHNOLOGY

[3] N. Sugawara, K. Oda, H. Akaike, and H. Kakuda, SHoes, Realtime e-learning System for Classroom Education and Organizational Learning, Transactions of Information Processing Society of Japan, Vol.48, No.8, 2007, pp.2791-2801.

[4] Y. Hirai and A. Hazeyama, A Learning Support System based on Question-posing and Its Evaluation, Proceedings of the Fifth International Conference on Creating, Connecting and Collaborating through Computing (C5), 2007, pp.180-185.

[5] T. Hirashima, T. Yokoyama, M. Okamoto, A. Takeuchi, Learning by Problem-Posing as Sentence-Integration and Experimental Use, Proceedings of AIED2007, 2007, pp254-261.

[6] M. Takagi, M. Tanaka, and Y. Teshigawara, A Collaborative WBT System Enabling Students to Create Quizzes and to Review Them Interactively, Transactions of Information Processing Society of Japan, Vol.48, No.3, 2007, pp.1532-1545.

[7] D. Hoshino, M. Takagi, N. Minami, Y. Teshigawara, Navigation Function of Group Review for Promoting Collaborative Improvement of quizzes Created by Students, Proceedings of the Eighth IASTED International Conference on Web-Based Education, 2009, pp.353-359,

[8] M. Takagi, M. Mochizuki, M. Mochizuki, and Y. Teshigawara, Proposal and Practice of Exchange Learning through Quiz Creation and Peer Review, International Journal of informatics Society, 2010, to be published.

[9] M. Takagi, S. Sakabe, M. Mochizuki, and Y. Teshigawara, Application of a Web-based Learning System "CollabTest" for Quiz-creation to Courses and its Evaluation, Transactions of Japanese Society for Information and Systems in Education, Vol.27, No1, 2010, pp.53-66.

[10] KEEPAD INTERACTIVE, TurningPoint,

ISSN: 1792-5061

324

ISBN: 978-960-474-232-5

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download