Candidate’s Evidence of Research & Creative Activities



Candidate’s Evidence of Research & Creative Activities

Allan C. Jeong

In the field of instructional design and distance learning, interest in the pedagogical applications of computer-mediated communication (CMC) to support collaborative learning (CSCL) continues to grow with the ever-increasing array of social and communication technologies now available on the Internet. The significance of CMC and CSCL research has grown due to the increased use of networked computers and social software within education and the workplace, the rapid pace of change in today’s world, and the increasing reliance on collaborative work and problem-solving. Even though the advantages of CMC are commonly recognized by both researchers and practitioners (e.g., more time to read and compose substantive responses, reduces social inequalities by diminishing the salience of physical and social cues), we still have not achieved a full understanding of CMC and its potential because we have yet to examine how characteristics of the message, messenger, response, responder, collaborative tasks, and discussion environment affect the processes of critical discourse (e.g., argument-challenge-explain) and how changes in discourse processes ultimately affect learning. We also lack adequate tools and methods to sequentially analyze, computationally model, and visualize the complex discourse processes that support group learning, decision-making, and problem solving.

To meet these challenges, I have established a programmatic and integrated line of research centered on developing the next generation of software tools and methods to enable researchers to take a micro-genetic approach to analyzing social-cognitive processes in group discourse, measured in terms of message-response sequences that promote/inhibit critical thinking (e.g., claim(challenge(no response vs. simple agreement vs. counterchallenge vs. explanation). To this end, all of the papers I have published in peer-reviewed journals stem directly from the software tools (see “ForumManager” and “Discussion Analysis Tool” in section “Other Scholarly Activity) and computational algorithms I developed to address the questions examined within each paper.

In this statement, I describe how I have used ForumManager and Discussion Analysis Tool (DAT) to conduct programmatic and integrated research that seeks to: a) determine which and to what extent certain characteristics of the messenger (gender, intellectual openness, learning style), the message (message function, conversational vs. expository style, intensifiers vs. qualifiers, response time, day of posting), and instructional environment (prescribed conversational scripts and message tags, pre-structured/unstructured discussion threads, imposing constraints on message-reply sequences) help to elicit the types of responses or speech act sequences that produce and sustain critical inquiry; and b) produce visual diagrams of stochastic models to concisely convey how specific factors/characteristics affect the processes of critical discourse.

Overview of Scholarly Accomplishments

Before I describe my research in detail, here is a brief summary of my accomplishments. I have a total of 14 published papers or in press (8 single authored, 4 first author), two submitted papers (both single authored), and four manuscripts in progress (see section “Other Scholarly Activity”). Six of the papers are published in ISI listed journals – two of which are published in the number 1 and number 4 ranked educational journals (ranked on impact factor) listed in the ISI 2006 Social Science index. Five of my papers are published in journals of international stature. All of the papers I have completed here at FSU are the products a programmatic research agenda intended specifically to determine ways to measure and predict how students respond (or do not respond) to messages based on who, when, why, and how messages and responses are posted in online discussions. In addition, I have 22 refereed conference papers (plus four accepted papers), and have received three invitations to write book chapters (see “Chapter invitations” in section “Selected Unsolicited Letters”). These are all indications of how my work has achieved international recognition, and how researchers find my methods to be unique and an insightful approach to micro-analyzing where the action is and understanding how discourse supports learning.

Foundational Papers & Creative Works

My first publication, “Sequential analysis of group interaction and critical thinking in online threaded discussions” (Jeong, 2003), illustrates how DAT was used to sequentially analyze message-response sequences produced in online debates with business students in a business ethics course. This paper was published in the premier, highly esteemed, and internationally recognized journal in the field of distance education, the American Journal of Distance Education. It was ranked first among all distance learning journals in a 2004 survey conducted with faculty in Instructional Systems and sister programs. The articles in this particular journal (from 2001 to 2003) were cited most frequently (n =130) by nine other distance learning journals among all articles cited in the ten distance learning journals (based on a 2003 survey conducted by students in my online course, Introduction to Distance Learning). According to unsolicited emails from professionals, three invitations to write book chapters (see Selected Unsolicited Letters), and the papers of other researchers in the field of CMC and distance learning (see “Citation Counts” in Other Scholarly Activity), this and subsequent studies has been recognized as seminal work that presents a unique approach to gaining new and important insights on the role of social interactions in online discourse.

Another foundational paper, “A guide to analyzing message-response sequences and group interaction in computer-mediated communication” (Jeong, 2005), was published in Distance Education, a highly esteemed journal ranked third among the distance learning journals (2004 faculty survey) and ranked number 2 in number of times cited in nine other distance learning journals (n = 55) based on the 2003 survey. This paper describes how to use DAT to sequentially analyze and determine how characteristics of the message, messenger, responder, and instructional context help to trigger or elicit responses that either support/inhibit the critical discourse. The guidelines presented in this paper were formulated, refined, and based on the methods developed during the time I conducted studies on the effects of gender, conversational style, response time, and dialog scripts and message tagging - where students are restricted to posting specific types of messages and required to tag their postings to identify the primary function of each message and responses to messages (e.g., argument, challenge, explanation, evidence).

The software program I developed, the Discussion Analysis Tool, was enhanced and released to the general public in the summer of 2002 (see “DAT” in section “Other Scholarly Activity”). One of the main enhancements enables users to convert response probabilities (e.g., the percent of responses to arguments that are challenges vs. explanations vs. supporting evidence) into state diagrams to visually convey and provide a Gestalt view of response patterns produced in group discussions (see illustration on page 1). I was awarded the First-Year Assistant Professor research grant (sponsored by Florida State University) in 2002, which provided full summer salary to further develop the software. I was also awarded a COFRS summer grant (sponsored by FSU) in 2003, and the CERPS summer grant in 2006 (sponsored by the College of Education at FSU) to write NSF grant proposals to fund further software development. Additional enhancements were recently made to help users compute “mean response scores” in addition to response probabilities (e.g., mean number of challenges, explanations, and evidence elicited per argument).

I also developed ForumManager (see section “Other Scholarly Activity”) to download threaded discussions from the Blackboard course management system. Once discussions are downloaded, ForumManager can be used to assess group performance on criteria such as the percent of messages that elicited responses, number of threads, and the average length of discussion threads. It can also be used to assess individual performance on criteria such as the number of postings, replies, number of replies elicited by each students’ posting, number of times each student reciprocates a reply, number of posting days, and number of postings containing user-specified keywords like “but” and “however” (keywords that might serve as indicators of critical thinking). The program also pulls out the message tags (e.g., ARG, BUT, EXPL, EVID) inserted by students in the subject lines and records the position of each posting within a thread (thread level). This data can then be loaded into the DAT software to sequentially analyze and identify patterns in message-response exchanges.

Selected Papers

The three selected papers I describe below (see section “Selected Evidence”) are studies I have published in highly esteemed peer-reviewed journals. These papers best illustrate the potential value and insights gained from using sequential analysis (and DAT) to examine computer-mediated discourse. The first paper, “The effects of conversational styles of communication on group interaction patterns and argumentation in online discussions” (Jeong, 2006), is published in Instructional Science (ISI journal, ranked #4 on impact factor among the 100 educational journals listed in the ISI database, 30% acceptance rate, impact factor = 1.81). This paper demonstrates how DAT can be used to create computational models to help us understand how characteristics of the message text (the presence of conversational language like greetings, addressing participants by name, emoticons, and signatures when used in the absence of non-verbal cues) affect how likely or how often students respond to messages (with challenges or counterchallenges in particular) in ways that promote and sustain critical inquiry, and identifies the specific behaviors that instructors should encourage and discourage to raise the level of inquiry in group discussions (effect sizes ranging from .00 to +0.74). In a similar study (Jeong, 2005) where I examine the characteristics of the message, I compared the number of responses elicited by arguments when arguments were presented with qualifiers (e.g., I think, maybe) vs. intensifiers (e.g., always, never) vs. without qualifiers nor intensifiers (stated factually).

The second paper, Scaffolding Collaborative Argumentation in Asynchronous Discussions with Message Constraints and Message Labels (Jeong & Juong, 2007), is published in Computers & Education (ISI journal, impact factor = 1.085, 10% acceptance rate). This paper provides a thorough introduction to the use of dialogue scripts and message tags to facilitate online discourse. The study’s findings suggest that message tags alone do not improve and at times can hinder performance (effect sizes ranging from -.98 to -1.88). However, the real advantages of using message tags are achieved when the tags and software programs like DAT are used to assess and diagnose the quality of exchanges produced in online debates and threaded discussion boards. This paper also serves to illustrate some potential methods (message tagging) that can be used in online discussion to generate sufficiently large data sets needed to conduct sequential analysis, and how DAT can be used to investigate and gain deeper understanding of how dialogue scripts affect the dialogic processes that produce and sustain critical inquiry. Using the same approach, I have published three additional papers that examine the effects of the inherent characteristics of asynchronous threaded discussions - the effects of pre-structuring discussion threads (Brooks & Jeong, 2006), short vs. long response times (Jeong, 2004), and early-in-the-week vs. weekend postings (Jeong & Frazier, in press).

The third paper, “The effects of gender interaction patterns on student participation in computer-supported collaborative argumentation” (Jeong & Davidson-Shivers, 2006), is published in the journal Educational Technology, Research, and Development (ISI journal, 8% acceptance rate, impact factor = .364 in the ISI 2005 Social Science Edition). This journal was ranked the highest by 105 faculty within our field in 2005. This paper illustrates how my tools can be used to determine how characteristics of the messenger and responder (male vs. female) affect the types of message-response exchanges that are produced by participants in mostly male discussion groups. This study reveals important insights into the complex social-psychological nuances that shape social interactions, and insights into the types and frequencies of exchanges produced between participants of the same vs. opposing gender (reported effect sizes ranging from +0.12 to +0.32). I also completed three other papers that examine how characteristics of the messenger and responder affect social interactions. These studies examined the effects of gender in mostly female discussion groups (Jeong, 2006), intellectual openness (Jeong, in press), and active vs. reflective learning style (Jeong & Lee, 2007).

Synergy between Research and Teaching

I am fortunate that my area of research is also the area in which I teach. Large portions of the data examined in my investigations are generated from the courses I teach online. As a result, I have been able to apply what I learn from my investigations to make incremental improvements in the design and implementation of my online class discussions and activities in my online courses. In addition, I am using DAT and ForumManager in my courses to assess, diagnose, and improve my students’ ability to engage in more critical discourse in the weekly discussions and debates. Students in my online course, EME6635 Designing for Online Collaborative Learning, study and apply my research findings to design and develop their own computer-supported collaborative learning activities. My students have expressed to me that they find my class discussions to be better organized and more engaging than the discussions they have had in other online courses, and that the debates have been valuable in teaching them how to think more systematically and more critically when solving problems outside of class.

My research has also directly benefited nine doctoral and one master’s student who worked with me in research apprenticeships and directed independent studies. The collaborative work has resulted in five papers (three published, one in press, one in review), five peer-reviewed conference papers, and three accepted conference papers (see “Refereed article published” and “Refereed conference papers” in the Vitae). For example, one student in sports psychology worked with me to sequentially analyze dialog between doubles tennis partners to identify communication patterns that improve team performance and point/match outcomes. A student in mental health counseling used my tools to identify patterns in decision-making processes that determine how obese vs. non-obese subjects choose items from a McDonald menu. Another student in information studies will use my tools to identify and compare patterns in eye/gaze movements on web pages between Americans vs. Koreans. These projects demonstrate how my methods and tools can be used across disciplines.

Synergy between Research and Service

My unique approach to studying social interaction, and my recognition as a scholar in online learning and online discourse, has earned me a position in two editorial boards for Educational Technology, Research & Development (the most prominent journal among faculty and practitioners in the area of instructional design and instructional technology) and Quarterly Journal of Distance Learning. I also review manuscripts for seven other journals: Journal of Computer-Assisted Learning, Computers & Education, The American Journal of Distance Education, Tech Trends, Journal of Distance Learning, and IEEE Transactions on Professional Communication. In addition, I have been invited on several occasions to demonstrate my research and tools to distance technology support staff at Academic and Professional Services at FSU. At this time we are continuing to discuss possible ways to integrate my tools with the Blackboard course management system and disseminate the tools to teaching faculty here at FSU (see “APS letter” in “Other Scholarly Activity”).

Looking Ahead into the Future

Because sequential analysis has been claimed by some to be the missing factor in the quest to understand “interactivity” and the role of social interaction in human learning and behavior, I believe that my research will have broad applications across multiple disciplines – particularly in the areas of educational research, cognitive psychology, instructional technology, human-computer interaction, human communications, social psychology, and sociology. In addition, my research addresses some of the goals of NSF and its cyber-infrastructure initiative to develop data analysis and visualization tools to support educational research (see “Grant Proposal Pending” in section “Other Scholarly Activity”). My research also shares some of the goals of the UK e-Social Science initiative and the European interests in methods for analyzing CSCL, particularly the Kaleidoscope Network of Excellence, which involves all the major educational research labs in Europe. With respect to these goals, I look forward to applying my research to the large-scale development of cyber tools to support educational research and assessment.

___________________________________

Allan C. Jeong

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download