NJ METHODOLOGY CHAPTER
Chapter 3: Research Methodology
3.1 Introduction
This chapter presents the research methodology adopted in this study. It discusses the methodological approach which seems to be the most appropriate to the nature of the research topic and to the research questions. To begin with, the research questions and their respective rationales are presented. This is followed by the research paradigm (approach). Then, subsequent to research methods (techniques), details of the design of the three data collection instruments employed in this study and their procedure will be presented. Finally, details of the data collection and data analysis will be described.
3.2 Research questions
The aim of this study is to find out whether the approach and the methods of the KNC and Sisa material are appropriate for Korean Primary English instruction. On the basis of this aim, the present study, therefore, attempts to investigate four research questions. The first is focused on evaluating the approach and the methods KNC & Sisa material use for achieving the final aim, which is communicative competence, and concerns the ways the Sisa course embodies a communicative approach. The second concerns the two teachers’ interpretation and implementation of the KNC and the Sisa material in the classroom. My third and fourth, respectively, concern pupils’ opinions about the Sisa material and the lessons, and the language and the oral skills they develop from the Sisa course.
In what ways does the Sisa Course embody a communicative approach?
Sisa claims that it has chosen to focus on oral language within a communicative approach excluding written language in order to develop the learners’ communicative competence. It focuses more on listening than speaking for the first year and uses audio-visual material as the main resource for providing new items of language and supportive activities for practising in every class without giving direct explanation of the meaning and the structure of the language.
This approach appears to be heavily influenced by Krashen’s theories. Asher’s (1965) Total Physical Response (TPR) and Krashen & Terrell’s (1983) the Natural Approach (NA) put a greater emphasis on the primacy of comprehension in the classroom, and questioned students’ need to produce the language they are learning. Even though TPR and NA took slightly different approaches to listening (TPR: listen and respond, NA: listen until they speak naturally), both of them are based on an innatist language learning view which suggests that because of children’s LAD (Language Acquisition Device) or UG (Universal Grammar), language learning occurs simply by being exposed to samples of the target language.
In contrast to these views, the current version of the Communicative approach, following interactionist view, is based on the belief that language learning occurs through meaningful and contextualized interaction, not through mere exposure to a language. This meaningful interaction starts from infants’ babbling with their parents or caretakers (Rivers, 1981). Long (1985) argues that L2 can be learned through meaningful negotiation in which the learners seek clarification, confirmation, and repetition of L2 utterances they do not understand. Many researchers (Long, 1985; Pica, Young, and Doughty, 1987; Swain, 1985) also provide evidence in their research to support the assumption that teaching a language should be an interactive process between teachers and students, and among the students themselves.
The authors of the Sisa material indicated that they intend to lead the students to comprehend the language by using visual clues, such as pictures in the book and scenes from a video, and also by using children’s own experience and knowledge. The use of video materials in language classrooms, especially for developing oral language skills, is not a new approach. Questions about improving language learning through comprehending the language mainly by using visual clues from video programmes have been raised recently by some researchers (Fisher, 1984; Hintzman, 1993; Peters, 1983) who suggest that children learn language through focusing on words. The implication of the research is that it is important for the children to pay attention to language.
Given the Sisa’s claims to be developing communicative competence, it is important to investigate the way the Sisa material (textbook and video materials) structures the activities for achieving its main goal in the light of current theoretical perspectives.
How do the teachers interpret the approach underlying the Sisa course?
The Sisa material adopts an oral approach to language teaching which was new to the Korean teachers and learners since Korean primary schools mainly focus on written language. Therefore, if a heavily oral-based course is introduced in a context where teachers are used to a strong emphasis on literacy it is likely they will tend to re-interpret the course to fit in with their own beliefs and practices.
According to Tann (1993), ‘Personal Theory’ refers to a person’s set of beliefs, values, understandings, assumptions and ways of thinking about the teaching profession. Brown and McIntyre’s (1986) definition of ‘personal theory’ extends to one of ‘Professional Craft Knowledge’ which is:
• Derived from practical experience rather than formal training.
• Seldom made explicit.
• Related to the intuitive, spontaneous and routine aspects of teaching rather than to the more reflective and thoughtful activities in which teachers may engage at other times.
• Reflected in the ‘core professionalism’ of teachers and their ‘theories in use’ rather than their ‘extended professionalism’ and ‘espoused theories’.
• Embedded in, and tacitly guiding teachers’ everyday actions in the classroom.
It seems to imply then that a personal theory is a set of personal beliefs which are intuitive, routine and implicit and have a very strong influence on why teachers do what they do in the classroom. If it is assumed that beliefs underpin action, then beliefs are clearly an important means to understanding how and why people act the way they do (Zeichner et al, 1987).
However, as Argyris and Schon (1978) argue there is almost always a discrepancy between what professionals say they believe (their ‘espoused theories’) and the ways in which they act (their ‘theories in action’). This discrepancy, according to Schon (1978), occurs since the teachers bring their own experiential knowledge to the classroom, although it may not be articulated, and use it to make sense of both their own and their students’ behaviour.
I, therefore, wish to investigate the teachers’ perceptions of the Sisa course (about listening focused language learning, and the ways of presenting and practising the language) and the way they implement the course in the classroom. I will closely observe whether they are able to implement the Sisa material in the way it intends or whether they draw on earlier practice to cope with the problems found in the context. This investigation will provide one of the justifications for evaluating Sisa’s effectiveness and practicability in the Korean context. Furthermore, if we consider that the way teachers implement the course is what the course becomes to the learners, investigating the teachers’ perception of the course and the implications of those perceptions for the way they use materials in the classroom will be one of the essential factors for evaluation of the Sisa material. To conduct the evaluation, the following questions will be asked;
• What are the teachers’ perceptions of the Sisa course?
• What do the teachers do in teaching the language in the classroom?
• How do they present the language using visual clues?
• How do they practice the language using game like activities?
What do the pupils think of the Sisa course?
This question was asked on the assumption that the children will have various attitudes toward the new curriculum due to the different teaching and learning methods used in English and other subjects in the Korean primary schools. From the beginning of the school year, Korean primary children get used to written type of tasks, rote learning, examinations, and teacher-centred learning. It has been argued that the learners’ learning attitudes in the lessons to a large extent are affected by their prior learning experience (Jones, 2000). Therefore, as highlighted in Holliday (1994), without adjusting and reconciling the learners’ prior expectation of learning to the new approach, the possibility of achieving the main aim of the material will be reduced. Therefore, when the English lessons are carried out mainly using the oral language and game like activities, then, similar to the Korean teachers in this study, there are high possibilities that the learners’ perceptions and attitudes toward the Sisa oral approach will be different from their reactions to other subjects.
Another factor which may affect the learners’ perception of the English lessons will be the use of video material. In Korea it is well known that most children are familiar with watching television and video material for entertainment purpose. In a survey by KEDI in 1994, it was found that approximately 80% of primary schools in Seoul have experiences of watching English video programmes in the classroom without any follow up activity. Other studies show that children’s previous passive viewing experiences (for entertainment not for learning) of video could affect their actual perceptions and attitudes toward the educational programme (Bukingum, 1983). Therefore, it is possible that the Korean primary school children’s prior experience of watching videos may affect their attitude and perceptions toward the Sisa course and as a consequence may influence their language learning.
In my study, I, therefore, asked questions as follows:
• What do children think of the course?
• How do they respond and behave in the English lessons?
These will entail an investigation about how they react to the Sisa material and why they react like that in the classroom.
What language and oral skills do pupils develop from the Sisa course lesson?
There are 16 sets of situational dialogues both in the book and the video tapes. The text has been designed to present and practise one set of functional dialogues in each unit without changing the order of the dialogues nor extending the language. The opportunities of receiving or producing the language in different contexts is not provided in Sisa material. Therefore, in each unit, the children are supposed to listen and speak the same set of dialogues for four lesson hours and then they learn another set of dialogues in the next unit. Systematic recycling of learned language has not been found in Sisa. This means the learners are exposed to a very fixed, simplified, and scripted body of language in one academic year and as a consequence, the learners may not understand the language when it is presented differently from the text material. In addition to this, Sisa intends the learners to understand the language through visual clues from the video materials and pictures without giving explicit explanation of the language structure or any direct translation. Therefore, it is predicted that learners may learn language by chunks not by learning individual vocabulary items or by sentences.
Given the lack of evidence for the effectiveness of listening to fixed and heavily simplified input, (see the arguments in section 2.3.1 Chapter 2), it seemed important to investigate the nature of the language children might learn from the Sisa course. Therefore, I devised oral tasks to collect examples of both receptive and productive skills. These may reveal whether the learners memorized the language without fully understanding the individual vocabulary items or sentence structure or, whether they understand the language when it is used differently from the textbook and whether they can produce the language appropriately in different contexts. Answers to these questions will allow me to draw out implications about the advantages and disadvantages of providing limited and fixed input to the learners in the way that Sisa selected
3.3 The research paradigm (approach)
Research can be defined as a systematic enquiry with the aim of producing knowledge (Ernest, 1994, p. 8). That raises the importance of the enquiry in research, which may be classified by the form of enquiry it employs. Enquiries may be categorized in terms of their purposes; exploratory, descriptive, and explanatory (Robson, 1993, p. 42). There is no restriction on the number of the purposes a study might have, it might be any combination of these purposes. However, one of the purposes often predominates. When exploratory purposes are used in a study, by the researcher, what is happening is discovered, new insights are searched for and phenomena are assessed in a new light. This type of research is usually qualitative rather than quantitative. An explanation of a situation or problem, usually in the form of a causal relationship is what a researcher is searching for in research with an explanatory purpose. This type of research as well as research with descriptive purposes may be qualitative and/or quantitative. In research with descriptive purposes, an accurate profile of persons, events or situations are portrayed by the researcher. This purpose needs extensive previous knowledge of the situation to be researched or described, so that the researcher knows the appropriate aspects on which to gather information (ibid., p. 42). Descriptive and exploratory purposes predominate in this study: it is descriptive because it deals with perspectives about the Sisa text materials, it is exploratory because it tries to find out what is happening. It is not explanatory because it, in no way, seeks to provide explanations of cause and effect.
Before the methodological approach applied in this study is described and presented, I will briefly discuss how I decided on the paradigm that underpins this research (see p.54 in Romberg, 1992, and Ernest 1994 p.18). The relation between research methods and paradigms is discussed by Guba and Lincoln (1994, p.105) who think paradigms are superior to methods of enquiry in a research:
both qualitative and quantitative methods may be used appropriately with any research paradigm. Questions of methods are secondary to questions of paradigm, which we define as the basic belief system or world view that guides the investigator, not only in choices of method but in ontologically and epistemologically fundamental ways.
Deciding the paradigm is, nevertheless, a very difficult task so that they are sometimes so closely interrelated or even overlapping that it is difficult to establish clearly what the differences are. Blaikie (1993, p. 215) states these complexities and the limitation as follows:
In adopting an approach to social enquiry, the researcher is buying into a set of choices with far-reaching implications... No one approach or strategy, and its accompanying choices on these issues, provides a perfect solution for the researcher; there is no ideal way to gain knowledge of the social world....all involve assumptions, judgements and compromises; all are claimed to have deficiencies. However, depending on where one stands, it is possible to argue their relative merits.
In the light of the literature review, it has been observed that there are mainly two paradigms positivist; the empirical-analytical, logical positivist, behaviourist or positivist or scientific or normative (see Cohen and Manion, 1994) paradigm and non-positivist; the interpretive (or naturalistic, constructivist, alternative paradigms research and qualitative approach to educational research (see Ernest, 1994 and Robson, 1993) including hermeneutic, phenomenological or symbolic approaches and the critical approaches (Ernest 1994; Kilpatrick 1988; Romberg 1992; Vithal 1999). These two paradigms are also called normative and interpretive respectively (Cohen et al., 2000). ‘Behaviour’ is a key concept in normative paradigm whereas interpretative approaches focus on ‘action’. Going back to my research questions it can be seen that they are concerned with the interpretation of actions, and meanings placed on those actions. The positivist paradigm is criticised by Guba and Lincoln (1994, p. 106) who stated that:
Human behaviour, unlike that of physical objects, cannot be understood without reference to the meanings and purposes attached by human actors to their activities.
With a non-positivist base, this study has encompassed an interpretative perspective. The main concern of this paradigm is human understanding, interpretation and intersubjectivity and it uses an ethnographic case study, largely qualitative forms of enquiry and triangulation (Ernest 1994, p. 24). In recent years, this paradigm has gained much ground with the strong emergence of constructivism (Vithal, 1999). The reasons for my choosing a non-positivist paradigm can be justified both by the nature of the research project and by the research questions. My overall approach to this study may be called broadly or semi “ethnographic” (‘naturalistic’) (Cohen et al., ibid., p.137) in the sense that I have observed, as far as possible, ‘what is’ in both schools (Y and P primary schools in Korea) without manipulating the course of teaching and learning. So the study has been conducted in the original settings without any interfering action (Robson 1993, pp. 60-61). The intention of the study is to create as vivid a reconstruction as possible of the culture or groups being studied (LeCompte and Preissle, 1993, pp. 235). However, I have not provided the very full and detailed picture of the whole context that is naturally found in Korean primary school English lessons and so this study can only be termed semi-ethnographic. LeCompte and Preissle (1993) presented several key elements of the ethnographic research such as the extensive use of observational techniques, and sought a description and interpretation of ‘total phenomena’. Hitchock and Hughes (1989, pp. 52-53) suggests that ethnographic research also involves the description of the activities in relation to a particular cultural context from the point view of the members of that group themselves, the description and analysis of patterns of social interaction. They (ibid.,pp.39-44) also suggest that ethnographic research is concerned more with description rather than prediction, induction rather than deduction and construction rather than enumeration. Lincoln and Guba (1985, pp. 39-43) clarify the implications of the axioms of ethnographic paradigm and express that the natural mode of reporting is the case study.
Cohen et al. (2000, p.182) also state that the most suitable paradigm for the case study is the one that seeks to understand and interpret the world in terms of its actors and consequently may be described as interpretive and subjective. Nisbet and Watt (1984, p. 72) define case study as a specific instance which is frequently designed to illustrate a more general principle. Adelman et al. (1980 in Cohen et al.) sees it as the study of an instance in action. The single instance is a bounded-system, for example, a child, a clique, a class, a school, a community. The case study provides a unique example of real people in real situations so it helps the reader to understand ideas in a clear way rather than simply presenting them with abstract theories or principles (ibid., pp. 72-73).
The purpose of this research is to evaluate whether the Sisa material works well in the Korean context, and to assess the effect and effectiveness of the Sisa material. Robson (1993) names it as ‘Evaluation Research’ and suggests using a case study strategy because ‘the flexibility in design and execution of the case study together with the fact that most evaluations are concerned with the effectiveness and appropriateness of an innovation or programme in a specific setting’, make the case study strategy appropriate for many evaluations (p. 175).
Robson (1993) also defines a case study as a strategy for doing research which involves an empirical investigation of a particular contemporary phenomenon within its real life context using multiple sources of evidence (p. 52). He, then pointed out a few characteristics which are important. They are;
• A strategy, i. e. a stance or approach, rather than a method, such as observation or interviews.
• Concerned with research, taken in a broad sense and including, for example, evaluation.
• Empirical in the sense of relying on the collection of evidence about what is going on.
• About the particular; a study of that specific case ( the issue of what kind of generalization is possible from the case, and of how this might be done, will concern us greatly).
• Focused on a phenomenon in context, typically in situations where the boundary between the phenomenon and its context is not clear.
• Using multiple methods of evidence or data collection (p. 52)
To summarise the above mentioned points, the case study is an empirical research of a particular phenomenon in context using multiple methods of evidence or data collection.
The strength of a case study is that it is trying to get some feeling as to what is going on in a particular situation. That is we can have some information about the research in depth and detail and where our own impressions and perceptions about the processes we are studying can be brought out into the open (Robson, 1993, p.54). Another advantage of using case studies is their flexibility as compared with experimental designs which have pre-specified designs and any failure to carry out this design will have serious implications.
One of the disadvantages of case studies is that it frequently follows the interpretive tradition of research which views the situation through the eyes of participants (Cohen and Manion, 2000, p. 183). Therefore a criticism is that it treats peculiarities rather than regularities (Smith, 1991: 375 quoted in Cohen and Manion, 2000). Robson (1993) explained that in this approach, biased and selective accounts are undoubtedly possible, and suggested it is necessary to check the reliability and validity of the findings to overcome the subjective interpretation of the case. However, Cohen and Manion (ibid), argued that the peculiarities which may occur only once in the whole case study can have a significant impact on understanding the whole. Therefore, significance rather than frequency is a hall mark of case studies, offering the researcher an insight into the real dynamics of situations and people (Cohen and Manion, 2000, p. 185).
Research can be categorised in terms of the data the researcher is seeking. Quantitative research seeks to establish facts, make predictions, and test hypotheses that have already been stated by using a deductive approach. The data analysis of quantitative research is mainly statistical, striving to show that the world can be looked at in terms of one reality; this reality, when isolated in context, can be measured and understood, a perspective known as positivism (Gay and Airasian, 1999). In contrast to the statistical nature of quantitative research, “Qualitative research is multi-method in focus, involving an interpretative, naturalistic approach to its subject matter” (Denzin and Lincoln, 1994). This means “qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them” (Denzin and Lincoln, ibid.). So my research is qualitative.
In summary, this study has exploratory and descriptive purposes. It uses an anti-positivist paradigm which is interpretativist with a naturalistic, broadly ethnographic case study approach. In terms of the type of the data, my research is qualitative.
3.4 Research methods (techniques)
The choice of the method of inquiry depends on the research paradigm. The research methods on the base of different paradigms might be different. Bogdan and Taylor (1984, p. 2) suggest that the researchers using these two paradigms undertake different kinds of problems and search for different kinds of answers, therefore their research requires different methodologies. These differences might be pertinent to what these paradigms are concerned with. I have chosen a non-positivist paradigm, which is concerned with human understanding and interpretation.
In an anti-positivist approach mostly qualitative methods are used although quantitative methods could be used as well. Participant observation, in-depth interviewing (Bogdan and Taylor 1984, p. 2), ethnographic case study, and triangulation (Ernest 1994, p. 24) are some of the methods used in anti-positivist research which yields rich, descriptive and contextually situated data (King, 1996).
After the theoretical framework of the research, the research questions can be answered through employing many appropriate methods. For this purpose, qualitative research using multiple methods are drawn on in this study. This approach has many advantages even though it is time consuming. The initial and obvious benefit of using a multiple-method approach is that it involves more data, thus being likely to improve the quality of the research (Denscombe, 1998). The multiple-method approach also reduces inappropriate certainty. That is to say, finding a definite result by using a single method may mislead the researcher into believing that s/he has found the ‘right’ answer. Using additional methods may point to conflicting answers, which remove specious certainty (Robson 1993, p. 290). They may help to answer complementary questions rather then specific questions. They may also be used in enhancing interpretability. For instance in a quantitative study, statistical analysis can be enhanced by interviews or a narrative account (Robson 1993, pp. 290-291). Because evaluating a course is very complex this requires multiple ways of collecting data so as to give a full picture of how the course is implemented in the context. Therefore, in this study, the appropriate methods are selected in terms of the characteristics of my research and research paradigm. The research methods used in this study are document analysis, which is evaluation of materials, and also interviews, observation and task based oral tests which I now describe.
3.4.1 Evaluation of materials
General rationale of Evaluation
According to Murphy (1994), although the reason for evaluating programme and texts will depend on the purpose of the evaluation, in general, people do the evaluation to know how the programme or the text works and how worthwhile it is (Murphy, ibid.). It has been argued that there is no predetermined correct theory of language teaching, which originated from second language acquisition research (Cook 1996) and, fits every language programme. Therefore, evaluation is vital (McDonough and Shaw, 2003). Nevertheless, since there is no definite yardstick to evaluate the course or the material (McDonough and Shaw, 2003; Sheldon, 1988), we have to make relevant judgements taking account of the learning context, the type of the learner for whom the course book would be suitable, age, interests and objectives of the course, size of the class, and equipment. (Cunningsworth, 1984; Tomlinson, 1998). In this next part, therefore, I will briefly describe the materials evaluation framework that I wish to adopt in my study, based on the review of different evaluation schemes carried out in Chapter 2. I will begin by clarifying the different types of evaluation which are available.
Predictive and retrospective evaluation
Ellis (1997), suggests a framework of task evaluation. He said that to examine the effectiveness of the task in the classroom, the researchers/ teachers should do two kinds of evaluation; predictive and retrospective.
A predictive evaluation, he said , can be called ‘external evaluation’ which can be carried out theoretically (i.e. by determining whether the assumptions that task designers make when they design specific tasks are justified in the light of some theory of language acquisition or skill development). He also stated that the published work on materials evaluation (Breen and Candlin, 1987; Cunningsworth, 1984; McDonough and Shaw, 2003; Skierso, 1991) reveals that it is almost entirely concerned with predictive evaluation because it seeks to determine whether materials are likely to work in a specific teaching context.
Ellis (ibid.) states that in retrospective evaluation, there are three different kinds of evaluation (see Figure 3.1 below): a student-based evaluation, a response-based evaluation, and a learning-based evaluation. A student evaluation provides information about how interesting and useful learners perceive a task to be. In other words, this evaluation can be carried out by interview or questionnaire to the students. A response-evaluation is about how the learner responds to the question that the task requires them to solve and the question directly relate to learners’ actual learning language. A learning-based evaluation is to find out whether the task actually contributed to the learners’ second language proficiency (see Figure 3.1)
With reference to Ellis (1997), my evaluation comprises two parts: one part of it is a predictive evaluation in which I examine the Sisa course proposals in the light of current language learning theories. The second part involves a retrospective evaluation in which I investigate the learners’ and the teachers’ responses to the course and whether the learner responses to materials resulted in their learning as intended by the Sisa material. The definition of the predictive evaluation used by Ellis and others (Breen and Candlin, 1987; Cunningsworth, 1984; McDonough and Shaw, 2003; Skierso 1991) is to evaluate the material or task before using it in the field. The evaluation of the Sisa material may not strictly be called a ‘predictive evaluation’ since the course has been used for one and half years (a period of evaluation) in Korean primary schools. However, as the evaluator, I wished to asses the quality of the textbook based on language learning theories before I actually observed its use in the classroom. Therefore, I called it a ‘predictive’ evaluation. It is also a retrospective evaluation since I am investigating actual teacher and learner responses to the material carried out through observation and interviews. A student-based and a response based evaluation, was carried out by both informal and formal interviews with pupils (instead of questionnaire as Ellis (ibid.) suggested), and a task based oral test at the end of the term.
Figure 3.1: Types of task evaluation (Ellis, 1997, p. 222).
3.4.2 Interviews
Although evaluation of the Sisa materials may help to identify and analyse the aims (communicative competence in terms of the language learning theories), it does not show anything about how teachers use it and how pupils learn from it. So the evaluation of the materials is followed up with interviews in order to understand the teachers’ and students’ views and perceptions regarding the use of the Sisa material and English language teaching in the Korean context.
It has been pointed out that the interview is a kind of conversation; a conversation with a purpose (Robson, 1993). This purposeful interaction has always been viewed as obtaining research-relevant information, which is focused by the researcher based on specified research objectives (Cannell and Kahn, 1968). Interviews help to find out from people those things which cannot be directly observed. In this connection, Kerlinger (1970) suggests that it might be used to follow up unexpected results, for example, or to validate other methods, or to go deeper into the motivations of respondents and their reasons for responding as they do.
Interviews are a very widely used instrument in social research. Using interviews in research is critical to gathering deeper information about the responses given by students or teachers. Cohen et al. (2000) have given three purposes for using interviews in research; to gather information having direct bearing on the research objectives, testing a new hypothesis or suggesting a new hypothesis and using interviews in conjunction with other methods (multiple-method, see also Dyer, 1995, p. 64). There are many different types of interviews which are given by Cohen et al. (2000, p. 270) in detail. The choosing of an appropriate type of interview depends on the purpose of the inquiry. I will focus on the types of interviews which are distinguished by a commonly used typology.
Dyer (1995, pp. 58-59) sees different forms of interviews as lying along a continuum. Structured and unstructured interviews are at the two ends of this continuum. Structured interviews have an accurate form and direction of the questioning determined in advance of actually meeting for the interview. At the other end of the continuum, there is no prepared list of questions. The interviewer is free to ask questions depending on the information received from the informant. Dyer points out that a combination of these two forms, which is called a semi-structured format, is probably the most successful approach to use in interviewing.
Robson (2002, p. 270) explicitly categorised interviews under three titles; fully structured, semi structured and unstructured interviews based on their structure and standardisations. These are identical with the use of Dyer (ibid.). Semi-structured and unstructured interviews are referred to as qualitative research interviews by King (1994 cited in Robson, 2002). These interviews are most appropriate for exploratory purposes in a research enquiry.
So the most appropriate interview method for this study seems to be a semi-structured interview. However, within the semi-structured interview, I have applied Tomlinson’s (1989) hierarchical focusing interview method. There are some advantages which distinguish the hierarchical focusing interview from the structured and unstructured interview methods. The hierarchical interview provides the interviewer flexibility to change the wording or makes it possible to eliminate some questions in line with the conversation flow as the topic may be covered in responses to earlier questions even though both hierarchical and structured interviews determine the aims and required information to be gathered. Furthermore, open-ended questions are used in hierarchical interviews, starting with general questions and gradually moving to more specific questions with the aim of getting points the interviewer needs to address in the study (Drever, 1995). Starting with general questions allows the nervous interviewees to settle down and prepares them for the further specific questions in terms of comfort, speech, confidence and getting familiar with the interviewer. The hierarchical interview is pointed out as a suitable way to elicit the research data in an effective way by Novak et al. (1984) and Tomlinson (1989).
A number of problems appear to attend the use of the interview as a research technique. One of these is that of invalidity. According to Lansing, Ginsberg and Graaten (1968), the cause of invalidity is bias which they define as ‘ a systematic or persistent tendency to make errors in the same direction, that is, to overstate or understate the ‘ true value’ of an attribute’. That is researchers can lead the interviewee to provide the answer which the researcher wants instead of the interviewees providing their own ideas (Robson, 1993). Also researchers can mis-interpret the answers. One way of validating interview measures is to compare the interview measure with another measure that has already been shown to be valid (Cohen and Manion, 1995), a form of triangulation. Another way of ensuring reliability and cutting down on interviewer bias is by using hierarchical focusing. According to Tomlinson (1989), ‘hierarchical focusing’ is a strategy designed to achieve two aspirations held by virtually all interviewers: to get at what they want, but to influence the respondent’s perspectives as little as possible.
Another way of giving flexibility and a relaxing atmosphere for the interviewees is to have different kinds of interviews (informal/formal interviews, one to one/ one to group) at different times and places. Ebutt and Watts, (1987 in Cohen and Manion, 1994), have considered the advantage of group interviewing as a means of collecting data in educational research. The advantages the authors identify is the potential for discussions to develop. Lewis (1992) suggests some advantages for using group interviews with children. For example, Lewis (ibid) found it useful to understand the learning difficulties of 10 year olds and she noted that the children challenged and extended each others’ ideas in the group interview. The group interview is also acknowledged as it collects people’s varied opinions and it is less intimidating for children than individual interviews (Cohen and Manion, 2000). Simon (1982) and Lewis (1992) chart some difficulties in interviewing children in terms of how to behave as a researcher and how to encourage the children to focus on the topic etc. They require the physical layout of the room to be not too big or small, so that everyone can see everyone. Group size is also a cause for concern. Lewis’s (ibid.) research indicates that a group of around six or seven is an optimum size for the children around age 10, though it can be smaller for younger children. As regards the disadvantages of group interviews, Watts and Ebbutt (1987) note that there are strong possibilities that the responses to questions from some group members only might be followed up, ignoring others. In both P and Y primary schools, there was a boy and a girl who always dominated the opinions of others during the group interviews as indicated by Watts and Ebbutt (ibid.). Due to these children, the data about the children’s perception of Sisa might have been influenced. This is maybe one of the disadvantages of group interview.
3.4.3 Observation
In the case of this study, observation of the classroom will be a significant method of data generation and data collection. In the case of this particular study, it is not enough for me to interview teachers and learners about their perceptions of the Sisa material or the implementation of this in the classroom. It may be that the beliefs of teachers and pupils and what they actually say and do do not always match. So this needs to be investigated as discrepancies between the two can reveal new insights.
Observation is a research instrument that allows researchers to gather ‘live’ data from the ‘live’ situations and in which the investigator systematically watches, listens and records the phenomenon of interest. Observation is used to validate or corroborate the information obtained in teachers’ and students’ interviews and evaluation of teachers’ use of material. So it can be used as a supportive or supplementary data technique. It helps to address different but complementary research questions rather than focusing on a single specific research question. Observation is a distinct and direct way to collect data. It is more direct than asking people about their views, feelings or attitudes or obtaining what people say they do, or what they say they think. It is watching people to witness events first hand, to find out what they do and to listen to what they say (Robson, 1993).
A major advantage of observation as a technique is its directness. The researchers can probe deeply and analyse intensively the multifarious phenomena that constitute the life cycle of a unit with a view to establishing generalisations about the wider population to which that unit belongs (Cohen and Manion, 1994). This information will provide the data that will tell the researcher the discrepancy between what is said by the interviewees and their actual behaviour. In my research, the observation technique will provide first hand experience where I can understand the reality of the classroom situation and will give a holistic view of how Sisa is being implemented.
Morrison (1993, p. 80) argued that by using observation the researcher gathers data on physical settings, the human settings, the interactional setting and the programme setting. The teachers’ teaching approach to and the implementation of the Sisa materials can be observed as well as the students response to the Sisa material and teaching so as to have complementary data on the research questions in addition to the main instruments which are the evaluation of the textbooks and the interview data collected from both the teachers and students.
An advantage of participant observations is that you can collect more detailed data than with any other research method (Burgess, 1984). A disadvantage could be that the observer when participating in the situation too much cannot find any time to write down the events on the spot (Robson, 1993). Non- participant observers, on the other hand, stand aloof from the group activities they are investigating and eschew group membership, in other words, the researcher sitting at the back of a classroom coding up frequently (Cohen and Manion, 1994). To be a real non-participant observer in the natural settings might be very difficult unless the observer can be an invisible person or have a separate detached room with a glass between the two rooms where the observer could observe the class without being noticed by the observants. If it is a children’s classroom, not being involved might be particularly difficult.
Bailley (1978) argues that most studies in a natural setting are unstructured participant observation studies while the opposite is true in an artificial environment. Cohen and Manion (1994) added that there is tendency that the unstructured, ethnographic accounts of work are the most typical method of observation in the natural surroundings of the school and the structured inventories of study habits and personality employed in the artificial setting. These unstructured and structured observation can be dichotomized as Formal or Informal information gathering. Informal approaches allow the observer considerable freedom in gathering the data. They would use note-taking and diary-keeping as tools. This kind of data collection can provide the observer with complex and difficult tasks to synthesize the collected information. Formal approaches tend to have very pre-specified aspects. From this approach, high reliability and validity can be easily achieved but at the cost of a loss of complexity and completeness by comparison with the informal approach (Robson, 1993).
In my opinion, it is difficult to use entirely unstructured or very structured methods in a particular context. This is because in the former case, the researcher will always have some assumptions in his/her mind which will influence what he/she observes. In the latter case of highly structured observation, the researcher may miss other interesting and relevant data. In this study, I decided to use semi-structured observation since everything happening in a classroom cannot be written down. Therefore, we need to have some themes and rough categories in mind to focus on in collecting the data.
The observer needs to be aware of the limitations and bias of using semi-structured observation. Spada (1994) pointed out the limitations of focused observation according to two factors. The first one is this orientation seriously limits and restricts the observer’s perceptions and the second one is that observation schemes tend to treat classroom language as discrete and isolated instances of verbal behaviour rather than extended discourse. This means that, in practice, the continued use of certain types of structured observation schedule could inhibit your participation in the group. Also it creates a kind of tunnel vision because the observer sees only those types of behaviour that coincide with the categories in the observation scheme. This restriction, it is claimed, “results in a failure to observe other behaviour that may be equally important features of the teaching and learning environment” (Spada, 1994, p. 686).
To alleviate the problems of observation, and also to have validity, reliability, and generalisability, in this study I will use descriptive field-notes as well as audiotaped recordings. Hence, not only is triangulation of instruments used but also triangular techniques are suitable when a holistic view of educational outcomes is sought (Cohen and Manion, 1994).
3.4.4 Task based oral assessment
For the study, task based oral tests were conducted with children after the interviews and the observations. The reasons for doing this is that I want to identify what language (vocabulary and structure) and language skills (listening comprehension and speaking) children have acquired from the lesson either from the Sisa material oriented or teachers’ reorganized lesson. The purpose of the research is to investigate how the Sisa material works in the Korean context. So the test is one way of investigating effectiveness in terms of pupils’ language learning.
According to Ellis (1997), this type of assessment is essential for evaluating the language programme to find out if it works actually as it was intended to. Ellis (1997) named this kind of evaluation as learning-based evaluation but also mentioned that it is very difficult to carry out. The tester should be aware of the learners’ previous knowledge of the language before they conduct the assessment so that they know if the learners’ knowledge results from the performing of the tasks in the programme. To avoid this problem, I will try to select the children who have never studied English outside the classes on a regular basis.
Spada (1994) argued that in the interaction analysis approach or any other approach to classroom observation, it is simply not enough for researchers to describe what goes on in classrooms only using one research method, either product oriented (a method depending on the outcomes of the learning) or process oriented (observation of classroom). She stated that it is beneficial for learning to use both process and product variables, because, in a pure ‘ product study’ the researcher focuses on comparing learning outcomes, often in the form of test scores without documenting what actually went on in the classrooms. In a pure ‘process study’ the researcher describes what went on in the classrooms, by collecting and analysing interaction-based data, but does not document the learning outcomes. In a ‘process-product study’ the researcher tries specifically to describe the classroom processes and relate them to documented learning outcomes which is essential for evaluating the effectiveness of the language learning programmes (Long, 1980). So in this study, I intend to focus both on classroom processes as well as learning outcomes.
3.5 Sample
In order to increase the quality of research, both appropriateness of paradigm and research methods and also the suitability of the sampling strategy are important (Cohen et al. 2000, p. 92). Sample size and sampling methods are two important issues in research.
There are two basic approaches to sampling: probabilistic and non-probabilistic. If the purpose of the research is to draw conclusions or make predictions affecting the population as a whole (as most research usually is), then a probabilistic sampling approach is used. On the other hand, if the research is interested in only seeing how a small group, perhaps even a representative group, is doing for purposes of illustration or explanation, then a non-probabilistic sampling approach is used. The sampling approach in this study will be non-probabilistic. The types of sampling are convenience, quota, purposeful, dimensional and snowball (Cohen et al. 2000, p. 104).
Purposeful sampling is a dominant strategy in qualitative research. Purposeful sampling seeks information-rich cases which can be studied in depth (Patton, 1990, p. 182-183). Patton identifies and describes 16 types of purposeful sampling (Patton, 1990, p. 169-183). The most appropriate purposeful sampling strategy for my study is convenience sampling in which available individuals are taken or the cases are taken as they occur.
As this study intends to use a case study, I do not necessarily go for a big sample but a small sized sample as the focus is on depth and not breadth. Also, throughout the process of formulating this study, I aware that the study itself is already complicated. Therefore, I selected two primary schools to be investigated from which the research will gain more in depth data and detail.
The schools are selected from the cities located near the Seoul area where the living standard and education are average for Korean cities. For the observation, two English classes of two primary schools were selected as a sample. The percentage of mixed students of high and low level of English in classes was not considered, because of the difficulties of finding classes with a balanced spread and the practical difficulties involved in testing for this. There were many positive variables to control, for example, rural vs urban, well vs badly equipped, student proportion of high and low level of language proficiency in one class, different communicative competences of the teachers, but given the limited time frame and the difficulties of gaining access, the sample was restricted to two classes in two Korean primary schools. Two third grade English classes of two state primary schools in Anyang were selected in the second term of the 1999 education season. Anyang is one of the oldest satellite cities in Korea. In this city, there is a newly developed area which is called Pyungchon. In this new region, there were many newly built apartments and schools, and the apartments were grouped by their size. Compared to the old area of Anyang, every facility, for example, road, hospitals, shops and private institutes, in Pyungchon was better and newer. So the price of these apartments were higher than the other area of Anyang and therefore, the people’s standard of living in Pyungchon was higher than those of Anyang.
P primary school is located in a group of middle sized apartments in Pyungchon. Therefore, the people’s standard of living was average in Pyungchon but higher than the people who were living in the other areas in Anyang. Next to P primary school, there was a high school which was rated highly in the whole Anyang city and there were many private institutes including private English learning institutes near to the school. The school itself was also newly built and all the equipment and decorations were new and well organised. The school had a separate English classroom which had a wide screen TV and a Video player, and an audio tape player. There were 35 students in W’s class and one of them had lived in the USA for 5 years.
Y primary school is located in the east end of Anyang in which many old and small sized apartments were gathered. Therefore, the people’s standard of living was slightly lower than the people who were living in Pyungchon. In the area near to the school, there were a few private institutes but not as many as in the area of P primary school. The school building did not seem to be newly built, but it looked as if it has been well looked after. The English classroom was used for music lessons as well and had a wide screen TV monitor and a video player, and an audio tape player. There were 38 students in J’s class and different from P primary school, there was not a single child who had lived abroad. Many children, according to J, the English teacher, (80%) had experiences in learning English out of school and the many of them (45%) were having English lessons at home through daily work sheets. Detailed information of teacher J is described in Chapter 5.
For interviews, two teachers (W and J) and 12 children (6 children from each school) from P and Y primary schools participated. For the oral assessment, the same children who participated in the interviews were assessed. In Chapter 5, detailed information of the samples, the two schools, the two teachers, and the learners who especially participated in the interview and oral tasks, will be provided.
3.6 Ethical Issues
In researching social science, it has been argued that we have to strike a balance between the demands placed on them as professional scientists in pursuit of truth, and their subjects’ rights and values potentially threatened by the research (Cohen et al., 2003). That is, social scientists generally have a responsibility not only to their profession in its search for knowledge and quest for truth, but also for the subjects they depend on for their work (p. 56). Therefore, being ethical limits the choices we can make in the pursuit of truth since respect for human dignity is more important (Cavan, 1977). Balancing these two issues in research have been accompanied by a growing awareness of the attendant moral issues implicit in the work of social researchers and of their need to meet their obligations with respect to those involved in, or affected by, their investigation (Cohen and Manion, 2003). With regard to the issues, I will discuss how I consider and apply these in my thesis in this section.
Cohen and Manion (2003) state the followings as the key issues that should be considered for seeking ethical issues in social researches; Informed consent, Access and acceptance, Voices of experience, Ethical dilemmas such as privacy, anonymity, confidentiality, betrayal, deception. In this thesis, I have addressed some of these issues (informed consent, access and acceptance, the nature of ethics in social research, matters of privacy, anonymity, confidentiality, betrayal and deception) that are relevant to my research. It is because that each research undertaking is different and so that the issues to be considered are not the same in each context. Cohen and Manion, (2003) explain that;
Each research undertaking is different and investigators may find that on one occasion their work proceeds smoothly without the Hydraheaded creature of ethical concern breaking surface. At another time, they may come to realize that, suddenly and without prior indication, they are in the middle of an ethical minefield, and that the residual problems of a technical and administrative nature that one expects as a matter of course when pursuing educational research are compounded by unforeseen moral questions. (p. 49)
Informed consent
Cohen and Manion (2003) state that much social research need to obtain the consent and co-operation of subjects who are to assist in investigations and of significant others in the institutions or organizations providing the research facilities (p. 50- p. 51). Informed consent has been defined by Diener and Crandall (1978) as ‘the procedures in which individuals choose whether to participate in an investigation after being informed of facts that would be likely to influence their decisions’. This definition involves four elements: competence, voluntarism, full information and comprehension. ‘Competence’ implies that responsible, mature individuals will make correct decisions if they are given the relevant information. ‘Voluntarism’ entails applying the principle of informed consent and thus ensuring that participants freely choose to take part (or not) in the research and guarantees that exposure to risks is undertaken knowingly and voluntarily. ‘Full information’ implies that consent is fully informed, though in practice it is often impossible for researchers to inform subjects on everything. ‘Comprehension’ refers to the fact that participants fully understand the nature of the research project, even when procedures are complicated and entail risks. Diener and Crandall (1978) argue that if these four elements are present, researchers can be assured that subjects’ rights will have been given appropriate consideration (p. 51).
In my thesis these four elements of informed consent had seemingly been pursued both from the teachers and the children. Firstly, the participants of this thesis were two English teachers and the pupils of the two teachers from two different schools in Anyang city. The two teachers were competent to make the proper decisions about their participation. They were given voluntarism for participating in my research. I told them the reason, purpose and the methods of the research before they made a decision. They seemed to comprehend the research purpose and the methods that would be used for my research since it was explained to them in Korean and in detail.
This informed consent can be subject to the people who might have equal relationships between researchers. Therefore, in some educational researches, like mine, which involves children who cannot be regarded as being an equal terms with the researchers, it is suggested to seek informed consent in two stages. First, researchers consult and seek permission from those adults responsible for the prospective subjects; and, second, they approach the young people themselves (Cohen and Manion, 2003). In the latter stage, as is clearly indicated in Fine and Sadstrom (1998), children are subjects to be explained about the research taking account of their level of understanding.
In this thesis, the voluntarism of children was seemingly not entirely pursued in terms of observation. The children of the two schools did not have full information about the research and the reason for their being observed. After gaining permissions on the observation from the teachers, they allowed me to be in their lessons and told the children that they will have an observant in their English lessons for a certain period of time. Due to the cultural norms of Korean society, it was not necessary to inform the children’s parents. In other words, it was acceptable in Korean society since the observation was carried out during their school hours and the two English teachers were in charge of their classroom students during their English lessons. Therefore, children’s voluntary decision of being observed by an unfamiliar adult was not considered.
On the part of both of the teachers, here was a little bit of confusion with regard to the participation of children in the case of the oral interview tasks. One of the teachers assumed her authority and control and made decisions about who would participate in the interviews. It is important to once again note that this was in-keeping with the context of the Korean role of a teacher which can extend beyond the normal boundaries of the school yard walls.
Since the oral interviews had to be done after school hours, firstly, I tried to give them full information about the interview and the OA (Oral Assessment). Then I confirm whether they did comprehend the given information. After the confirmation of their comprehension, secondly, I attempted to get voluntary decision from the children and the permission from the parents whose children were participating in the interview and OA. For instance, before the observation was started and one week prior to take the OA, I told the two teachers about the purpose, the ways of doing the interview and oral tasks, and the time that would take for accomplishing these tasks. I asked the two teachers to explain the children about the interview and the OA and then ask them for voluntary participation. Then from the volunteers, select 6 students who are in the middle group in other subject areas. I also asked the teachers to get a permission from the parents of these children. However, the children’s voluntarism seemed not to be pursuit in the pilot interview and OA. It maybe because that the teacher perceived that assessing children’s learning from the lessons might be one way of measuring his own teaching ability. It could also have been because of his cultural perception of his role. Therefore, he chose 6 children who were in the higher group in English language rather than asking their voluntary participation. In Korean culture, teachers’ request is equivalent to an order which students would not easily reject. In this sense, I thought the children’s right of voluntarism was seemingly not achieved. It should be noted that in general children do not have the right to voluntarism for most of their learning tasks.
The experience that I had with the pilot OA cited above helped me to establish culturally responsive methods for guaranteeing more informed participation for the real OA. In the real OA, I asked the two teachers to give the children more freedom of making their own voluntary decision of participation as well as permissions from their parents in advance. I confirmed again at the time of OA and interview whether they voluntarily participated and whether they get the permission from the parents, and the information about the interview and the OA. Whereas all the children in P school got the permission of staying in the school for two more hours after school, two children in Y school did not get their permissions from their parents. So I asked them to go out and make a phone call to their parents for the permission.
I started to explain about the interview and OA after the two children got their permissions from their parents. Then I found out that the children were not informed well about the interview and OA. All the children of the two schools did not have correct information. They understood that they would have an English test. Therefore, I explained them again about the interview (I said that we will have a little talk over snacks and drinks about school) and OA (we will do a few activities). Then again, I gave them one more chance to make their own decision about taking participation in the tasks. All the children said that they understood about things they will do and they all cheerfully decided to do the interview and OA. Therefore, it can be said that the children had an informed consent about the research they participated in. This will be explained more in detail in the next section of this chapter.
Access and Acceptance
Cohen and Manion (2003) highlight the importance of gaining the access to the institution or organization where the research is to be conducted, and acceptance by those whose permission one needs before embarking on the task at the initial stage of the research project. They explain that it is relevant to the principle of informed consent because it offers the best opportunity for researchers to present their credentials as serious investigators and establish their own ethical position with respect to their proposed research.
They suggest three stages. The first is the gaining of official permission to undertake one’s research in the target community before the actual meeting with the people. This means contacting, in person or in writing. In this stage, Cohen and Manion (2003) write that researchers need to clarify in their own minds the precise nature and scope of their research. In this procedure, they suggest to identify the aims of the research, its practical applications, the design, methods and procedures to be used; the nature and size of samples or groups; what test are to be administered and how; what activities are to be observed; what subjects are to be interviewed; observational needs; the time involved; the degree of disruption envisaged; arrangements to guarantee confidentiality with respect to data---(see more about this in Cohen and Manion, 2003, p. 54).
Second is making actual contact in person. After an introductory letter, with appropriate people in the organization with a view to negotiating access (p. 55). Third is a negotiation process. At this point researchers will give as much information about the aims, nature and procedures of the research as is appropriate. This is very important; information that may prejudice the results of the investigation should be withheld (p. 55).
For my thesis, I followed these three stages. At the first stage, although it was not necessary to send a formal letter to the school to provide an informed consent about the research, I made phone calls to 4 schools in Anyang city that were using Sisa textbook and material at the time of this research. I was informed about these schools from Korea Ministry of Education. I gave them some information about myself as a PhD student and the purpose of this study briefly to the vice head teachers of the schools. Only two schools were available while the other two did not have interests in my research. The vice head teachers of these two schools kindly requested me to make a personal contact with the English teachers.
So, as the second stage, I again told them about myself as a PhD student, the aim and the purpose of my research briefly through the phone. I said that I want to see how they understand the new English textbook and how they implement this in their own lessons. These two English teachers were interested in the research and allowed me to visit them. Also they agreed on having interviews with me at the first visit.
As the third stage, I visited them to their English classrooms at off lesson hours. When I met them, I again gave brief information about myself and about the aims and research methods that will be used for this research. I also told them about the duration of this observation and the activities that will be used with the children. The two teachers willingly accepted everything I requested.
We conclude this section by reminding beginning researchers in particular that there will be times when ethical considerations will pervade much of their work and that these will be no more so than at the stage of access and acceptance, where appropriateness of topic, design, methods, guarantees of confidentiality, analysis and dissemination of findings must be negotiated with relative openness, sensitivity, honesty, accuracy and scientific impartiality. As we have indicated earlier, there can be no rigid rules in this context. It will be a case of formulating and abiding by one’s own situational ethics. These will determine what is acceptable and what is not acceptable (p. 56).
Voices of Experience
Baumrind (1964), for example, warns of the possible failure on the researchers’ part to perceive a positive indebtedness to their subjects for their services, perhaps, she suggests, because the detachment which investigators bring to their task prevents appreciation of subjects as individuals. Cohen and Manion (2003) explain that this kind of omission can be averted if the experimenters are prepared to spend a few minutes with subjects afterwards in order to thank them for their participation, answer their questions, reassure them that they did well, and generally talk to them for a time (p. 59).
In this thesis, I tried to thank them at almost every classroom observations and the interviews both with the teachers and the children. Sometimes, I brought some cookies or flowers to the teachers as another way of expressing my appreciation toward them. Occasionally, I had a tea or coffee with them after the observation to thank them and to ask some questions that appeared spontaneously during the lessons. As is common in Korea, I took the teachers to the meal to appreciate their support and participation. As time goes by, thanks to all these appreciation, we became more friendly and it affected my observation. The teachers seemed to feel more comfortable seeing me in their lessons and looked more natural when they are teaching.
As Cohen and Manion (2003) clearly indicated, I tried to concern for the welfare of participants, especially children (p. 60). For instance, as I have already mentioned earlier in this section, at the time of having the OA, I prepared some snacks and juices for them children in order to thank their participation and to relax their nervousness of being assessed by an adult. After the OA, I also paid them a deep appreciation about their participation and their performing well.
It is also highlighted to make a formal statement of acknowledging and thanking the participants support and help in whatever form of written account, even to the extent of identifying by name those whose contribution was significant (Cohen and Manion, 2003). I, therefore, would like to make a formal appreciation about their support and help in my thesis.
Privacy
Anonymity and confidentiality are considered as one of methods which can protect the privacy of participants (Cohen and Manion, 2003). The essence of anonymity is that information provided by participants should in no way reveal their identity. A participant or subject is therefore considered anonymous when the researcher or another person cannot identify the participant or subject from the information provided. A subject agreeing to a face-to-face interview, on the other hand, can in no way expect anonymity. At most, the interviewer can promise confidentiality. Non-traceability is an important matter, and this extends to aggregating data in some cases, so that an individual’s response is not identifiable. The principal means of ensuring anonymity then, is not using the names of the participants or any other personal means of identification. (p. 62)
In order to protect the participants’ (the two teachers and 12 children) privacy, as Aronson and Carlsmith (1969) note, the research data and all the names are transferred to coded and unnamed (Ex, Teacher W and J, S1 and S2). As they comment, ‘the very impersonality of the process is a great advantage ethically because it eliminates some of the negative consequences of the invasion of privacy (Aronson and Carlsmith, 1969: 33).
As Cohen and Manion, (2003) The second way of protecting a participant’s right to privacy is through the promise of confidentiality. For this, I promised the two teachers that I would not make any connection between the information I gathered and their names publicly. Even though they were not keen on the confidentiality and the anonymity, I told them about this from the beginning of the observation and confirmed it time to time during the observation.
3.7 Design of research instruments
As described earlier, the research methods chosen for this study are document analysis, interview, observation and oral tasks. A literature review was undertaken to find instruments used in similar studies and then to design the most appropriate instruments for this research. Methods and methodologies used in other studies which evaluate materials and language courses, and their connections with the research questions of this study are analysed throughout the literature review.
Table 3.1: Linking research instruments to sample and research questions.
|Research Questions|Sample |Instruments |
|RQ1 |Sisa materials |Evaluation |
|RQ2 |Teachers |Interview |
| | |Observation |
|RQ3 |Students |Interview |
| | |Observation |
|RQ4 |Students |Interview |
| | |Observation |
| | |Task oral based test |
Then, research instruments to be used in the study are created. The ways in which the Sisa course embodies a communicative approach, the ways teachers interpret the approach underlying the Sisa course, pupils’ thoughts about the Sisa course in the lesson and the language/oral skills the children develop from the Sisa course lesson are to be explored through evaluation of the Sisa material, interviews, observation and task based oral assessment. Using many instruments produces different kinds of data on the same topic. The initial and obvious benefit of using the multiple method is that it involves more data, thus being likely to improve the quality of the research (Denscombe, 1998) and it reduces inappropriate certainty. For the purpose of a quick overview, the research questions, and the relevant instruments are presented in Table 3.1. In the remaining section I will present the design of instruments with respect to the research questions and sample.
3.7.1 Evaluation
Evaluation framework I adapted based for this study is discussed in detail in Chapter 2.
3.7.2 Interview
A more complete understanding of teachers’ and students’ views and perceptions regarding the Sisa materials may be gained through interaction with teachers and students. The rationale for the interview method employed in this study is to gain a deeper understanding of teachers’ approach to the Sisa materials and pupils’ attitudes to the Sisa materials. Both interviews with teachers and students were conducted in Korea and these were held in three different periods (before, during, and after observation of the classes) to revalidate and reinvestigate the data collected from interviews as Kerlinger (1970) suggested. This was also to look at the differences between teachers’ and pupils’ perceptions of the course and their behaviours in relation to the language teaching/learning. The way of collecting the interview data at three different periods will be described in detail in the data collection section.
Teacher interviews (before observation)
For the study, I kept a few points in my mind to gather valid data from interviews. Firstly, for the teacher interviews, I made focused interview questions in order to obtain relevant data for the study. Hierarchical focusing was carried out in the process of interviewing to ensure that all the interviewees are asked the same questions. I, therefore, started off with fairly general questions that would put the interviewee at ease and then slowly focused on more specific questions or issues that came to mind. Starting with general questions allows the nervousness of the interviewees to calm down and provides me with insight into their personalities, allowing adjustment to accommodate the respondents’ idiosyncrasies.
There were six questions in the interview and each question has sub-questions about (see Appendix 3-1);
• General perceptions of language learning.
• General perceptions of the Sisa course.
• Sisa textbook.
• Using the Sisa video and audio material in the lesson.
• Using games and activities for learning language.
• The learners’ language learning.
The first was about teachers’ opinions of teaching English to Korean primary school children and this was followed by their perception of the proper age to start teaching language. The second question was about their general perceptions of the Sisa course which includes Sisa textbook and video material. Then the questions began to focus on their perceptions of using the video and audio material and with this, their opinions about oral focused language learning and the way Sisa presents L2 input was also asked. The more explicit and detailed question was about their ideas of using games and activities. This question was followed by their perception of practising the language in the Sisa material. The final question focused on the two teachers’ perceptions of the learners’ language learning through the Sisa course.
As Tomlinson (1989) suggested, a construal interview agenda was prepared to be in my hand during the interview with the aim of managing the interview, changing wording if necessary, and organising questions as a way of getting rid of the ones already answered.
Teacher interviews (before observation)
This interview was usually conducted right after the lessons. I asked some questions that occurred to me during the lessons. The main aim of having the interview in this period was to understand the following aspects (see appendix 3-3 for more information);
• teachers’ reasons for doing a certain activity
• teachers’ perceptions about children’s understanding of the activity
• teachers’ perceptions about children’s learning from the lessons
Pupil Interview (after observation)
One (interviewer) to group (interviewees) interviews were used with students since I felt that the children were more likely to talk more in the group interview than in the one to one situation. There are four questions and each question has sub-questions about general perceptions of (see Appendix 3-2);
• English language learning in the school.
• The Sisa material.
• The Sisa video.
• The Sisa activities and games.
These questions were framed appropriately to the children’s cognitive level as Lewis (1992) suggests. The first was about pupils’ opinions of English language learning in the school. This question was asked them as ‘what is your favourite subject?’. When they did not say English as their favourite subject, I asked them whether they like English lessons. The second question was about their opinions about Sisa material. Then the questions moved on to asking about their perceptions of video, activities and games. As shown in appendix 5-00, more sub-questions were asked in order to gain more in depth data. The second one to one informal interview with the teachers and the students also were used during the observation, to gather instinctive and spontaneous reactions to the activities and their teaching/learning strategies.
3.7.3 Observation
I used semi-structured observation for this study. I, therefore, planned to observe the following aspects of the classroom which may be able to shed further light on the issues under consideration.
• Whether or not the teachers follow the Teacher’s Guide (TG)
• Whether or not the teachers follow what they believe and say.
• How the pupils behave in the lessons they have which could be different from the ways of learning other subjects in the school
As well as to gain data about whether what teachers say is actually what they do, observation was also conducted to enrich the data on teachers. I aimed to have a non-participant role in the classroom, and to see as ‘normal’ teacher and student behaviour as possible. However, this was not entirely feasible since my presence may have affected the behaviour of the students and teachers. This was revealed more during the first two weeks’ observation when they were watching a Sisa video in which I was presenting material. Some students looked at me sitting in the very back of the classroom every time I showed on the TV screen, therefore, the teachers had to gather their attention back to watching the video.
I used a systematic observation with time-interval sampling to observe teachers in the classroom when they were teaching. A time-table was prepared for this purpose. The time interval was five minutes. Everything that occurred in the classroom in terms of teachers and teaching was manually recorded by the researcher. In this way, neither teachers nor students were influenced by the existence of a video camera.
3.7.4 Task based Oral Assessment
In this section, the goals and the method of both the Pilot and the Real OA will be described.
Description of research questions
The following questions were asked in order to find out how the Sisa material benefits the children’s language learning in the English classes of P and Y. They are:
• What language did the student learn from the lesson?
• How much did they remember of the language they previously learned without having systematic recycling of the learned language?
• Did they learn the language by memorising the chunks or by understanding individual words and sentences?
• To what extent did they apply the language they learned in a different context from textbook?
Four tasks for the Pilot OA and Five tasks for the Real OA were devised for collecting data for the above mentioned purposes, regarding the features of the Sisa material and the teaching methods the teachers of both schools used in their own classes.
Pilot Oral Task
The pilot OA consists of four oral tasks and one preparation activity as shown in Table 3.2 below. The preparation activity aims to provide preparation time for the testee to use L2 in the OA. Informal types of interview both in L1 and L2 was conducted using a puppet. The aims of tasks A and B were to find out how much the children comprehend the language they hear and how much they remember from what they had learnt. For task A, ‘listen and pick the right picture activity’ from ‘your turn 1’(Appendix 7-1) of each chapter was selected. For task B, directives were used to see how they respond toward the language they learned in a different context. It was, therefore, a ‘Listen and Do’ activity. Tasks C and D were devised to evaluate the production ability of the children using the language they learned from the text. In task C, direct translation, both from English to Korean and from Korean to English, was required of the children. In task D, role-play was planned to assess how well the children could use the memorized chunks in a situation where they had to decide what to say, how to say it and when to say it appropriately for the situation.
Table 3.2: Descriptions of the Tasks of Pilot OA.
|Tasks |Aims |Methods |
|A |Assessing Listening comprehension |Used ‘Your Turn 1’, which is ‘Listen and Pick the right picture’ type |
| | |activity, from each chapter. |
|B |Assessing Listening comprehension in the |Used ‘Listen and Do’ activity using directives. |
| |different context from the text | |
|C |Assessing the production ability |Used direct translation both from English to Korean and from Korean to |
| | |English. |
|D |Assessing the production ability in the |Used a role-play between myself and a child. |
| |different context from the text. | |
Researchers (Carpenter, Fujii and Kataoka; 1995) have used techniques like the use of puppets, the use of toy box in order to reduce the problem of avoidance and shyness on the part of children speaking with strange adults.
In this section, description of the Real OA will not be included since this is discussed in detail and in depth in Chapter 7.
3.7.5 Selection of sample for case study
The main focus of this study is to evaluate the Sisa material in terms of its efficiency and effectiveness of implementing it in the Korean context. The teachers’ and the children’s perception of the Sisa material and the English lessons as well as the way the teachers’ interpret and implement the Sisa material were investigated. Therefore the participants in this study were the teachers and the students (9-10 year olds) of the two schools (P and Y).
3.8 Data Collection
Data was collected by administering all research instruments to the designated sample in Korea. In this section, the way in which this data was collected with respect to students and teachers respectively is presented.
3.8.1 Evaluation of the Sisa material
Following the nature of retrospective evaluation of Ellis (1997), the Sisa textbook evaluation was carried out independently before the actual research for this study was started. The evaluation of the Sisa material was based on a framework adapted from both Littlejohn (1998) and McDonough and Shaw (2003). This framework is described in section 2.6 of Chapter 2.
3.8.2 Interview
Before observation
Formal interviews with the two teachers were conducted before the observation in order to gather the information about their perception of English lessons and of the Sisa course and materials. The interviews were held in their English classrooms after school hours before the observation. Before the interviews were started, the aim of this study and the interview were explained in detail to the teachers. They were told that they can say whatever they wanted to say. They were also told about the use of an audio recorder and note taking in advance of the interview. After the warming up and preparation section, the interview with the teachers were conducted by me. The teachers were seemingly relaxed in expressing their opinions and often they covered the other questions prepared for the interviews, without my asking. I did not interrupt them. However, I had to try to find out the next question to be asked. The teachers were not disturbed by the time I spent searching for the next question and they willingly answered all the questions. Each interview took about an hour and a half.
During observation
During the observation, the questions which occurred to me during the lesson were asked either on the spot or after the lesson. I asked some questions to the pupils and teachers spontaneously about their attitudes and responses to the activities after lessons. There was no certain form of questions prepared beforehand. Through these informal interviews, I could attain immediate and spontaneous personal perceptions of the activities and lessons. Also teachers’ interpretation or self explanation of their (teachers’/pupils’) behaviour and implementation were asked and gathered (see appendix 3-3).
After observation
The formal interviews with the children of the two schools were conducted after observation. The questions were carefully prepared taking into consideration the factors which occurred during the observations and my research questions. These were formal interviews that needed sets of prepared questions and an appointment fixed in advance with the interviewees, and the use of a tape recorder during the interviews. All the questions which arose during the observations were included in a Semi-structured questions. The detailed information about the interviews procedure were described in the section 5.4 of Chapter 5.
3.8.3 Observation
The English lessons of the two teachers chosen for the interviews were observed for 12 weeks (48 lessons) in this study. The teachers were again told the aim of the study and asked to teach in the way they always taught. I, therefore, planned to observe the following aspects of the classroom in order to help me answer my research questions as discussed earlier.
• The extent to which the teachers modify the Sisa material in the lessons.
• The degree to which their beliefs and perceptions are reflected in their lessons.
• The lesson structure of the teachers.
• The pupils’ attitudes in the lesson.
In relation to the first issue, I focused on the ways and the extent the Sisa material was modifed in their lessons. This issue was highlighted due to the Sisa approach (oral focused learning and the use of video and audio material, and game like activities) which is new and contradicts the traditional Korean educational approach. Hence, I predicted that the teachers may modify the Sisa material based on their beliefs of learning and teaching which was established through their own experience of learning and teaching. The second issue was elicited under the assumption that the teachers do not always follow what they believe in their practice. Therefore, in the observation, I will look at how much their beliefs match their own practice. The third issue was to be investigated in order to look at whether or not they follow the basic structure of Sisa or in what way the teachers formulate their lesson structure. The fourth issue was highlighted since I anticipated that the pupils may behave differently from the other subjects due to the different approach adopted in the Sisa course.
Duration of observation
I intended to observe two classes for at least 12 weeks. This duration was necessary for the following reasons. Firstly, two classes of two primary schools is quite a small sample as representative of Korean English classes. More detail and in depth information was needed to provide validity to the study. Also this duration was necessary when considering that one unit of the Sisa material is designed to be taught in two weeks’ time and there are only two hour long English lessons in a week. Therefore, in 12 weeks I could observe the learners learning 6 units.
Secondly, it was necessary to ensure that the atmosphere in the classroom was normal. It was because I used to be a TV presenter in my country that my presence in the school might provoke excitement. I needed some time to get used to the people in the school and for the children and the teachers to get used to me, so that I could gather more natural information. For two weeks, I decided not to do any actual note-taking or audio taping but saw how the people in the school behave. Also in this time, I could feel and see what is happening with an open mind to reduce bias and tunnel vision from my research questions.
On the first day of observation, I was introduced to the children by the teachers. The purposes and the reasons of my being there were introduced to them. As was anticipated, my presence in the schools made the children excited. In the first week, the children lined up for my signature every break time and during the lesson hours they kept on looking at me even though I was sitting at the very back of the classroom. The teachers had to make an effort to keep the children’s attention toward the lessons. I tried to get to know them and not to disturb them. I did not do any note taking, tape recording or interviews during the lessons at this period. These were done after the lessons.
The children started to get used to being with me and began to behave normally in the lessons. Active observation, therefore, was started. One audio tape recorder and note-taking were used. The field notes were shown to the teachers to reassure them that I was not evaluating the teachers but describing the class without personal judgement. However, the teachers did not want to see it and showed their trust in what I said to them. Spontaneous and informal one to one interviews with children were taken after the lessons.
During the observation, I collected both low and high inference behaviour. Low inference behaviour is something which can be easily interpreted and analysed in terms of the rationale and the purpose of certain behaviour. For example, how often the teacher gives direct translations to the pupils, or when the pupils ask the teacher about the meaning of the language would be an example of a low inference descriptor. High inference behaviour on the other hand, is one requiring the observer to make inferences about the observed behaviour such as ‘students’ lack of interest’ when they were doing listening activities. The problem with the latter is that it requires the researcher to infer unobservable mental states from observable behaviour and there is a danger that it can be misinterpreted. Moreover it is a skill that requires training and practice.
The observation data provided important ideas about the teachers’ teaching approach, lesson structure, and resources they used in the lesson. In the observed lessons, I was a semi-participant observant in the classroom. Similar to a non-participant observant, she sat at the very back of the classroom to be out of the students’ sight and prevent them from being distracted by her presence which could affect the teachers’ performance. However, she focused on specific issues when taking notes. The teaching approaches and everything the teachers did in the classroom were both tape-recorded and scripted manually by me. This way of recording the observation gave an immediate and fresh account and a full picture of the teachers at the time of observation in the lesson for me.
3.8.4 Task based Oral Assessment
The way of conducting the POA and ROA is presented in Chapter 7 in detail.
3.9 Data Analysis
After data collection, the next phase of research was concerned with data analysis. The data analysis technique was chosen based on the research design, research instruments and the method of data collection.
Table 3.3: Data and corresponding sample and instruments.
| |Sample and corresponded research instruments |
|Data form |Students |Teachers |Documents |
|Written accounts |- Oral task based test |- Interview |Evaluation of textbook |
| | |- Observation | |
|(Tape recorded and transcribed |- Interview |- Interview | |
|account)spoken words |- Observation |- Observation | |
In this study, the research instruments used in data collection yielded mostly qualitative data which were in the form of written accounts or spoken words (see Table 3.3). I had to use qualitative data analysis to make sense of this data in terms of the written accounts of teachers, students and documents about the situation, noting patterns, themes, categories and regularities (Cohen et al. 2000, p. 147).
3.9.1 Evaluation of the materials
The Sisa material was evaluated in terms of how it matches with what it claims to do and how it reflects the current theories of children’s foreign/second language learning. The Sisa input, topics, and activities were major aspects that were evaluated according to the criteria for this study. The Sisa input was evaluated according to the degree of authenticity and simplicity of input. I intended to look at how natural the Sisa input is and how useful it is for the learners’ language learning. I also analysed the appropriateness of Sisa L2 input in terms of level of difficulty for the learners’ linguistic level. The Sisa topic was analysed according to the degree of providing new and interesting information for the learners so that they can easily apply these topics to their own lives and promote their desire to talk in L2. The Sisa activities were investigated as to whether these provide opportunities for the learners to develop their communicative competence. Communicative competence was broadly divided into two aspects; functional and structural competence. To develop these competences, current theory suggests that both ‘familarity’ and ‘processing’ types of tasks are needed. The Sisa activities, therefore, were evaluated in the light of including these two task type activities, and interactiveness and meaningfulness.
3.9.2 Interviews
After the interviews with the teachers and students, each one was transcribed since it was conducted before the observation and then translated into English. After finishing transcribing both interviews with the teachers and the students, supportive data from the interviews was extracted to answer research questions and used to expand findings from the textbook evaluation. Quotations from the interviews were used to show and clarify the findings of textbook evaluation which could not be interpreted simply by analysing them according to the elicited criteria. In analysing the interviews, all transcriptions were read several times by me to capture the themes in the transcriptions, pertinent to the research questions. It was also aimed to categorise interview data for the purpose of organising to use them as quotations and supportive data for the other data (e.g., textbook evaluation, observation, OA). The interview data were mainly categorised according to the following categories;
• Perceptions of teachers towards the Sisa textbook and video material.
-Teachers’ perceptions of listening focused language learning
-Teachers’ perceptions of using video material as language input and presentation.
-Teachers’ perceptions of the Sisa game-like activities and songs.
• Teachers’ perceptions of the learners’ attitude toward language learning.
The analysis of interview data with the learners focused on the following themes/categories
• Learners’ perceptions of the Sisa text and the video materials.
-Learners’ opinions of focusing on listening comprehension
-Learners’ perceptions of game-like activities and songs.
-Learners’ perceptions of language classroom compared with the other subjects.
These were used to look at the match or mismatch between teachers’ perceptions and the reality of what pupils did in lessons.
By using the suggestions of Silverman (1993), to improve and increase the reliability of the interview, the interview schedule was carefully planned. To enhance the reliability of the categories, I cunducted the interview three different periods (before, during and after observation)and recorded the interview data using two different methods (audio taping and noting). The notion of valid interview data is problematic because the students’ social and school experience and the existence of multiple influences on their experiences might affect their answers in the interviews. But to try to overcome this problem, interview data was triangulated with observation and oral assessment data. Typically, interview data are considered valid when triangulation confirms that what the different parties say about an event coincide (Partington, 1998). Here, consistency of statements over time and recording method could be matched with data gained from observation and tests.
3.9.3 Observation
I mainly used descriptive field notes and audio taped recordings to collect the interactions between teachers and pupils, pupils and pupils, and teachers’ discourse in the lesson.
The field notes were mainly narrative description of the classroom situation but inevitably focused on the issues of the study. These issues were categories for the data analysis. The audio-taped data were transcribed and categorised. Hopkins (1985) stated that even though the method has been criticised by its time consumption since it can provide a very specific and accurate record of a limited aspect of the teaching, or a particular interaction, it is one of the most popular teacher research methods.
3.9.4 Task based Oral Assessment
All the procedures of OA were recorded by a tape recorder and were transcribed. The transcribed data as well as written form of answer sheets (Task A and B) were analysed based on the main goal and purposes of each task. The outcomes of the oral tasks of both schools firstly were analysed respectively and secondly were compared between the two schools. The outcomes were analysed and interpreted taking the two teachers’ lesson structure and teaching style and their use of activities in lessons into consideration.
Some limitations and problems were found in identifying the learners’ listening (Task B) and speaking (Task D) performances due to the nature of the tasks. These problems made it difficult to analyse an individual child’s performance and therefore, I had to analyse the outcomes of OA by the overall language achievement of each school group. Despite the problems and limitations, some tentative answers to the research questions regarding the language and the skills pupils learned from the Sisa course were found.
Conclusion
This chapter discussed the methodological approach that was adopted in my study. It described the research paradigm (approach), followed by research methods (techniques), details of the design of the three data collection instruments employed in this study and their procedure, and details of the data collection and data analysis. In this chapter, evaluation frameworks adopted for the Sisa textbook analysis, interviews between the two teachers and the pupils, observation of the two English lessons of the two Korean primary schools, and the POA and ROA were described in light of the rationale, the nature and the procedure of collecting and analysing the data.
-----------------------
Predictive
Task evaluation
Retrospective
Student-based
Response-based (internal)
Learning-based (external)
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- strategic planning methodology pdf
- education methodology examples
- product management methodology pdf
- teaching methodology in education
- marketing methodology definition
- definition of methodology in education
- sample of research methodology format
- methodology definition example
- marketing methodology and approach
- chapter by chapter bible summary
- introduction to research methodology pdf
- methodology step by step