WordPress.com



AuthorsYearFull citationCountryContextParticipantsAgesText SampleGenresProficiency/progression judgmentAppel & Wood2016Appel, R., & Wood, D. (2016). Recurrent word combinations in EAP test-taker writing: differences between high- and low-proficiency levels. Language Assessment Quarterly, 13(1), 55-71.CanadaProficiency test (CAEL).Not specifiedNot specifiedCanadian Academic English Language Assessment (CAEL) test. All essays in the analysis were from a single test administration and based on the same topic. Unknown topic as it was still in use.Maximum 45 minutes. 1-3 pages.339 x 'lower level' texts (grades 1-4); 254 x 'upper-level (grades 5-9).Unknown genre as the prompt was still in use. However, the essay required test takers to use material (reading texts/lecture listening materials) from the other test components in synthesising and rationalising their thoughts on the topic. Graded on holistic 9-point scale by three raters using read-aloud protocol. Lower and upper level texts decided by the grade texts received. Texts awarded 10-40 points were classified as lower-level while texts that received a score of 50-90 points were classified as higher-level texts.Arnaud1992Arnaud, P. J. L. (1992). Objective Lexical and Grammatical Characteristics of L2 Written Compositions and the Validity of Separate-Component Tests. In P. J. L. Arnaud & H. Bejoint (Eds.), Vocabulary and Applied Linguistics. (pp. 133-145). London: Palgrave Macmillan.FranceUniversity50 first-year undergraduate students at a French university.Not specifiedTwo one-hour essays on a single essay. Essays were written 6-8 weeks apart. Time limit of 1 hour. Topic for the first essay: ‘What is wrong with French secondary education? What improvements do you suggest?". Topic for the second essay: 'Supposing all professions were open to you, which one would you choose? Which one are you actually hoping to enter?'. Essay 1 mean word count: 327 and mean count of essay 2: 388 words. Not specified.Proficiency levels were decided by grammar and vocabulary test scores. Arthur(study 1)1979Arthur, B. (1979). Short-term changes in EFL composition skills. In C. A. Yorio, K. Perkins, & J. Schachter (Eds.), On TESOL ’79: The learner in focus (pp. 330-342). Washington DC: TESOL.USAUniversity preparation programme14 studentsLate teens – early twenties152 timed essays that were written over a 7-week period. Tasks included giving a short description of a short animation.Various including cause and effect type essaysProficiency was determined based on objective multiple-choice test scores in grammar, vocabulary, reading and oral comprehension skills. Longitudinally, texts written in the first half of the course (first six texts) contrasted with those in the second half of the course (last six texts)Arthur (study 2)1979USAUniversity preparation programme14 studentsLate teens – early twenties18 essays written by the 14 students in Arthur's first study. Essays were all on the same unknown topic. Various Texts ranked (with not specific criteria given) by 13 teachers.Aryadoust 2016Aryadoust, V. (2016). Understanding the growth of ESL paragraph writing skills and its relationships with linguistic features. Educational Psychology, 36(10), 1742-1770. SingaporeUniversity writing course116 x 1styear students on 16-week basic level English writing proficiency course.18-20 years6 texts from each participant: 2 pre-course; 2 mid-course; 2-post course. 180-230 words each. Maximum time allowed: 1.5 parison and contrast ExpositoryMarked by multiple raters, with grades given for content (20%), organisation (20%) and language (60%).Banerjee, Franceschina & Smith2007Banerjee, J., Franceschina, F., & Smith, A. M. (2007). Documenting features of written language production typical at different IELTS band score levels. IELTS Research Reports, 7, 1-69. China & South AmericaProficiency test (IELTS)Not specified16+ years275 writing scripts from IELTS tests (Task 1 and Task 2) written by test-takers with L1 Chinese or Spanish. Only bands 3-9 included in sample.Not specified.9-point scale. Grades awarded to live IELTS tests. Bardovi-Harlig1992Bardovi-Harlig, K. (1992). A second look at t-unit analysis: reconsidering the sentence. TESOL Quarterly, 26(2), 390-395.USAUniversity intensive English programme86 x students divided into 7 proficiency levels:56 x levels 1-630 x level 7Not specifiedTopic: How do you expect your life to be different from your grandparents'? Students were allocated 35 minutes to write their speculative essay. Not specified.Students divided into proficiency groups based on overall pass or non-pass on the placement test.Bardovi-Harlig & Bofman 1989Bardovi-Harlig, K., & Bofman, T. (1989). Attainment of syntactic and morphological accuracy by advanced language learners. Studies in Second Language Acquisition, 11, 17-34. USAUniversity language placement test30 x students divided into 2 equal-sized proficiency groups. All had achieved a TOEFL score of approximately 550 before taking the placement test.Not specifiedEssays were written as part of the university's placement test. Topics were non-technical and included giving advice to Americans visiting your country. Students had 45 minutes to answer 1 of 3 prompts.Various Students divided into two proficiency groups based on overall pass or non-pass on a placement test.Becker 2010Becker, A. (2010). Distinguishing linguistic and discourse features in ESL students' written performance. Modern Journal of Applied Linguistics, 2, 406-424.USAUniversity preparation programme43 1stsemester students on university intensive English program. Students from 3 levels (low-intermediate; high intermediate; advanced).Not specifiedWriting session from end-of-semester achievement test. Independent writing. 30 minutes allowed.Not specified.Graded on 6-point holistic scale (derived from analytic criteria related to language use, content and organization) by two raters. Scores from two raters summed.Bestgen2017Bestgen, Y. (2017). Beyond single-word measures: L2 writing assessment, lexical richness and formulaic competence. System, 69, 65-78.Not specifiedData set 1: Proficiency test (FCE)Data set 2: universityData set 1: Not specifiedData set 2: undergraduate studentsData set 1: Mostly (66%) 16-25 years Data set 2: 19-31 years (66% 21-25years)Data set 1: 1235 x First Certificate in English examination scripts (from Cambridge Learner Corpus). Each script includes 2 texts, total200-400 words. Max time allowed: 80 mins for both texts.Data set 2: 223 texts from ICLE (International Corpus of Learner English). 500-900 words each. 85% of text produced without time limit.Data set 1: Letter + either letter, composition, story, report or magazine articleData set 2: argumentative essaysData set 1: FCE score on scale from 0 to 40.Data set 2: graded by two raters against CEFR bands B1-C2 (11-point scale). IRR=.69. Where raters disagree by more than one band, sent to third rater. Final score is average of 2/3 grades.Bestgen & Granger2014Bestgen, Y., & Granger, S. (2014). Quantifying the development of phraseological competence in L2 English writing: An automated approach. Journal of Second Language Writing, 26, 28-41.USAUniversity intensive EAP programme57 students on Intensive English and EAP programmes at US universityNot specified171 timed essays (30 minutes each) elicited at three points across a 15-week semester, with 6-8 weeks between elicitations.Descriptive essays Graded by two raters using an analytic rubric with scores for content, organization, vocabulary, language use and mechanics. IRR, r=.88.Biber & Gray2013Biber, D., & Gray, B. (2013). Discourse characteristics of writing and speaking task types on the TOEFL iBT test: a lexico-grammatical analysis. TOEFL iBT Research Report, 19. Not specifiedProficiency test (TOEFL)Not specifiedNot specified960 essays in total. 480 Integrated and 480 independent essays. 30-minute time limit for the integrated essays and 20 minutes for the independent essays. Tasks had the purpose of giving personal opinions about life choices or general issues and in the integrated task summarise the content of provided textsIntegrated and independent essays 9-point holistic scale which was converted to an equivalent 4-point scale when comparisons with speaking tasks were made.Biber, Gray & Staples 2016Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37(5), 639-668.Not specifiedProficiency test (TOEFL)International TOEFL candidates of varying L1sNot specified947 writing texts in total: 476 Independent and 471 integrated writing TOEFL tasks. 30-minute time limit for the integrated essays and 20 minutes for the independent essays. Tasks had the purpose of giving personal opinions about life choices or general issues and in the integrated task summarise the content of provided texts.Integrated and independent essays9-point holistic scale which was converted to an equivalent 4-point scale when comparisons with speaking tasks were made.Bulté & Housen 2014Bulté, B., & Housen, A. (2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42-65.USAUniversity intensive EAP programme45 students on a one-semester intensive EAP programme.Not specifiedFirst and last essays written for writing programme; four months between essays.Descriptive essaysRating scales developed by Connor-Linton & Polio. Overall score (combining context, organization, language use, vocabulary, mechanics); language use score; vocabulary score. Cahyono, Mukminatien, & Amrina 2016Cahyono, B. Y., Mukminatien, N., & Amrina, R. (2016). Indonesian students' writing proficiency and their ability in using complex sentences. International Journal on Studies in English Language and Literature, 4(9), 22-32.IndonesiaUniversity argumentative writing course54 students in 2 classes:28 in Class A26 in Class BNot specified4 prompts: arranged marriage, censorship of books, songs and movies, laws against cell phone use while driving; prohibition of smoking in public places.Argumentative essaysEssays graded using the ESL Composition Profile rubric. Essays received a score from 1-100. This score determined proficiency levels.Casanave 1994Casanave, C. P. (1994). Language development in students' journals. Journal of Second Language Writing, 3(3), 179-201.JapanUniversity compulsory foreign language classes (UG)16 freshman/sophomore students from 2 English language modules.Not specifiedSampled from regular journal writing across three semesters. First 2 texts; 2 texts at end of first semester; 2 texts at end of third semester. 250-500 words each.Free-writing journal entriesTOEFL scores determined proficiency levels. Students were classed as intermediate level with a TOEFL score between 420-480.Castro2004Castro, C. D. (2004). Cohesion and the social construction of meaning in the essays of Filipino college students writing in L2 English. Asia Pacific Education Review, 5(2), 215-225. PhilippinesUniversity English class30 students from 3 intact classes at a private university. All students had Filipino as their L1. 15-19 yearsStudents were asked to write a take-home essay designed to prompt them to express their opinions, defendtheir stance, and include factual details and personal beliefs. The prompt was: Supporters of technology say that it solves problems and makes life better. Opponents argue that technology creates new problems that threaten and damage the quality of life. Using examples by which technology has improved or damaged modern living conditions, write an essay that discusses these two positions. Explain which view of technology you support and give reasons for your position. No time limit specified.Not specified2 raters used an 8 point holistic scale. The essays were then classifiedinto low (0.0 – 1.5), mid (2.0 – 2.5), andhigh (3.0 – 4.0) quality.Chen2017Chen, W. (2017). Profiling collocations in EFL writing of Chinese tertiary learners. RELC Journal, 50(1), 53-70.ChinaUniversity (UG)194 students in total:65 x 1st year students64 x 2nd year students65 x 3rd year studentsNot specifiedOne text from each student. Mean words per text 230-267. Written in class. ArgumentationsCross-sectional development studyChen & Baker2016Chen, Y. H., & Baker, P. (2016). Investigating Criterial Discourse Features across Second Language Development: Lexical Bundles in Rated Learner Essays, CEFR B1, B2 and C1. Applied Linguistics, 37(6), 849-880.Not specifiedNot specifiedNot specifiedNot specified1,029 essays from Longman Learner Corpus. After grading, reduced to:189 x CEFR B1239 x CEFR B2157 x CEFR C1Argumentative/expository essaysEach paper graded against CEFR by two raters (IRR=.844). 3rd rater used in cases of disagreement.Crossley, Kyle & McNamara 2016Crossley, S. A., Kyle, K., & McNamara, D. (2016). The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. Journal of Second Language Writing, 32, 1-16.USAUniversity intensive EAP programme57 students on upper-level EAP courseNot specifiedEach student wrote 3 essays (beginning, middle and end of semester): total 171 essays. Timed essays of 30 minutes. 4 sets of prompts. Average 335 words.Descriptive essays 5 feature analytical scale. Scores combined for overall mark. Two raters for each text. Study uses overall score (IRR=.88) and organization score (IRR=.70).Crossley & McNamara 2012Crossley, S. A., & McNamara, D. (2012). Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35(2), 115-135.China High school514 high school students. Not specified500-word essays written under exam conditions. Maximum time allowed: 1 hour, 15 minutes.Full corpus comprises 200 texts sampled from each of 6 assessment levels (A-F). Total of 1,200 texts. Study samples only texts between 485 and 555 words in length. 514 texts in total.EssaysGraded on a 6-point scale by trained raters from Hong Kong Examinations and Assessment Authority.Crossley & McNamara 2014Crossley, S. A., & McNamara, D. (2014). Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing, 26, 66-79.USAUniversity preparation programme57 L2 students attending university intensive writing classNot specified57 students produced 3 essays each: 171 essays in total. 30 minutes allocated for the essays with 4 sets of 2 prompts used across the essays. Average word length 335 words. Time limit: 30 minutes. Descriptive essays. 2 raters grade each text on 5-trait analytical scale. Scores summed to calculate total score. IRR for language use, r=.77; for total score r=.88Crossley, Salsbury & McNamara 2011Crossley, S. A., Salsbury, T., & McNamara, D. (2011). Predicting the proficiency level of language learners using lexical indices. Language Testing, 29(2), 243-263.USAUniversity preparation programme100 students from 19 different L1 backgrounds across 2 US universities. All students were studying on intensive English programmes (IEPs)17-34 years100 texts at beginner, intermediate and advanced proficiency levels (37 beginner level texts; 30 intermediate level texts; 33 advanced level texts). Analyses based on 150 word samples from each text to control for text length.Handwritten journal entries TOEFL and ACT ESL Compass scores.Cumming, Kantor, Baba, Erdosy, Eouanzoui & James2005Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10(1), 5-43.North America, Australia and Hong KongProficiency test (TOEFL)36 x students in pre-university or university-credit ESL courses at institutions in North America, Australia and Hong Kong. 12 students randomly selected from each of 3 band levels (bands 3-5). Only students whose writing received consistent grades by two raters on two tasks included in sample.Not specifiedTexts written as part of field work for new TOEFL tasks. For each participant:2 x independent essays (30 mins allowed); 2 x reading-writing tasks (25 mins allowed); 2 x listening x tasks (15 mins allowed). Total 216 texts Integrated and independent TOEFL essays.5-point scale. Each paper marked by 2 x ETS staff who regularly work on scoring TOEFL compositions.Cumming & Mellow 1996Cumming, A., & Mellow, J. D. (1996). An investigation into the validity of written indicators of second language proficiency. In A. Cumming & R. Berwick (Eds.), Validation in language testing (pp. 72-93). Clevedon, Philadelphia, Adelaide: Mutilingual Matters Ltd.CanadaUniversity preparation programme66 students divided into 2 groups:24 x Francophone students42 x Japanese studentsNot specifiedEach Francophone student wrote 3 essays based on:1) A letter describing their current course of study2) A summary of a 48-page popular science booklet about the functional anatomy of cats.3) An expository argument defending a position on women's social roles. Word limit not specifiedEach Japanese student wrote a single essay from a choice of 4 prompts:1) Should smoking be permitted in public places?2) Discuss the effects of your studies in Canada may have on your future3) Compare yourself to one of your parents4) Describe someone who has had a great influence on your life. Essay length was approximately 300 words. Francophone students: letters, summaries and argument essays. Japanese students: argument, cause/effect, description and comparisonFrench students were divided into intermediate and advanced proficiency groups by: scores on university intermediate test, class placement scores and length of residence in Canada. Japanese students were divided into proficiency groups based on their TOEFL scores. Advanced proficiency group: Mean TOEFL score: 512.8 (range: 473-577) and intermediate proficiency: Mean TOEFL score: 438.4 (range: 407-470).Daller, Turlik & Weir 2013Daller, H., Turlik, J., & Weir, I. (2013). Vocabulary acquisition and the learning curve. In S. Jarvis & H. Daller (Eds.), Vocabulary knowledge: Human ratings and automated measures (pp. 185-215). Amsterdam: John Benjamins.UAEUniversity foundation programme42 x students on university foundation programme. All are female UAE nationalsNot specified7 texts from each student. Written at 10-week intervals (80 weeks total) as part of foundation programme. Maximum time allowed: 1 hour 15 mins. Sample topic: What makes a good school?Descriptive and expository essays. IELTS 0-9 holistic bandsDastjerdi & Samian2011Dastjerdi, H. V., & Samian, S. H. (2011). Quality of Iranian EFL learners' argumentative essays: cohesive devices in focus. Mediterranean Journal of Social Sciences, 2(2), 65-76.IranPrivate language school40 non-English university major students enrolled in an English writing course 22-30 years40 essays on the topic: "Drinking a lot of water can help you to become healthier" Time limit: 25 minutes. Argumentative essays0-5 point TOEFL holistic scale used to rate essay by two raters. Inter-rater reliability across essays =.81. Mean of two scores taken as final score.De Haan & van Esch 2008De Haan, P., & Van Esch, K. (2008). Measuring and assessing the development of foreign language writing competence. Porta Linguarum, 9, 7-herlandsUniversity23 first year students18-19 years69 essays – students each wrote 3 essays. One essay per year across first three years of undergraduate study. Maximum time allowed: 30 mins. Topic: Preferred source of news and why.For analysis of cohesion, 9 texts selected. These sampled evenly across three proficiency bands attributed to students on the basis of their first essays. Argumentative essays9 essays selected for cohesion analysis ranked by four university teachers of English on the basis of holistic assessment. No rubric provided. Final score based on mean rank across raters.Douglas2015Douglas, S. R. (2015). The relationship between lexical frequency profiling measures and rater judgements of spoken and written general English language proficiency on the CELPIP-General Test. TESL Canada Journal (Special Issue 9), 43-64.CanadaProficiency test (CELPIP)CELPIP test candidatesNot specified802 essays across 2 tasks. Essay topics not specified. 20 text samples per CELPIP proficiency levels 3-12.Not specified.Exam papers previously rated by testing organizationEngber 1995Engber, C. A. (1995). The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing, 4(2), 139-155.USAUniversity preparation programme66 students on university intensive English program. Levels 4-7 of 7-level program: considered intermediate to high-intermediate.Not specified1 text per student. Timed essay portion of placement examination taken at end of each 7-week session. 35 minutes allowed. Topic: 'How will studying in the United States help your country'.Not specified.Initial proficiency level of intermediate decided by programme level. 6-point holistic scale based on Test of Written English criteria. Each essay marked by 10 raters. IRR based on 8 raters (highest/lowest removed): r=.93Espada-Gustilo 2011Espada-Gustilo, L. (2011). Linguistic features that impact essay scores: a corpus linguistic analysis of ESL writing in three proficiency levels. 3L: The Southeast Asian Journal of English Language Studies, 17(1), 55-64.PhilippinesUniversity150 freshman students across 5 private universitiesNot specified150 essays written in first week of classes. Time allowed: 1-1.5 hours. Students were instructed to write a single page of single-spaced text. "Write your ideas on the topic, The Importance of the English Language in the Different Areas of My Life: personal life, academic pursuit, and professional life. Provide a separate explanation for each area using examples and illustrations'.Not specified.Scored by three raters on 6-point holistic scale based on Test of Written English rubric. IRR=.71 Scores based on organization, language use, content and mechanics. For analysis, 6-point scale collapsed to 3-points.Ferris 1994Ferris, D. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28(2), 414-420.USAUniversity placement test160 ESL students. 40 x from four L1 groups: Arabic, Mandarin, Japanese, Spanish. Not specified160 x placement essays written in 35 minutes. Topic: effects of culture shockEffect essay Each text rated by three raters on 10-point scale. Summed to create score out of 30. Divided into high vs. low-level groups. Low group mean = 14.8/30; High group mean = 22.9/30.Flahive & Snow1980Flahive, D. E., & Snow, B. G. (1980). Measures of syntactic complexity in evaluating ESL compositions. In J. W. Oller & K. Perkins (Eds.), Research in language testing (pp. 171-176). Rowley, MA: Newbury House.USAUniversity50 students from each of 6 programme levels.Not specifiedSeveral topics used. Essays produced under carefully monitored conditions. 50-minute time limitExpository essays5-point holistic scale. Inter-rater reliability was above 90%Friginal, Li & Weigle 2014Friginal, E., Li, M., & Weigle, S. (2014). Revisiting multiple profiles of learner compositions: a comparison of highly rated NS and NNS essays. Journal of Second Language Writing, 23, 1-16.USAUniversity353 x graduate and undergraduate students from 9 universities across the USAAlso 150 native speaker undergraduates.18-38 years for NNSs.30 minute timed composition modelled on TOEFL iBT Independent Writing Task. Participants asked to discuss whether they agree with the statement "Nowadays, people put too much emphasis on personal appearance and fashion". Argumentative texts Ratings for the NNS essays were obtained from the original researcher. For the NS essays, two trained raters scored each essay following a rubric used in evaluating TOEFL iBT tests on a scale of 0–5. Only 24 NNS + 51 NS essays that received a score of 5 from both raters were selected for analysis.Friginal & Weigle2014Friginal, E., & Weigle, S. (2014). Exploring multiple profiles of L2 writing using multi-dimensional analysis. Journal of Second Language Writing, 26, 80-95.USAUniversity Intensive English programme & EAP programme.Students at Michigan State University.Not specified207 essays on topics(1): describe a good or bad teacher you've had (2) describe your current home. Average word count of each essay was 331 words.Each student writes 3 texts across one semester at 6-8 week intervals. 30 minutes for each text, with no access to outside resources.Descriptive essays Grading process described by Connor-Linton, J., & Polio, C. (2014). Comparing perspectives on L2 writing: Multiple analyses of a common corpus. Journal of second language writing, 26, 1-9. Texts graded independently by two raters.Essays grouped into low, mid and high quality by overall scores and also scores on vocabulary and general language use. Garner2016Garner, J. (2016). A phrase-frame approach to investigating phraseology in learner writing across proficiency levels. International Journal of Learner Corpus Research, 2(1), 31-68.Not specified (but participants have German nationality)Education First private language schoolsStudents completing online Englishtown coursesNot specified28,472 essays from EFCAMDAT corpus. Texts are writing tasks completed at the end of each online lesson. Corpus may include multiple texts from individual writers.Various Classes are offered at 16 proficiency levels. Judgements on level are not made explicit. Garner, Crossley & Kyle2018Garner, J., Crossley, S. A., & Kyle, K. (2018). Beginning and intermediate L2 writers' use of N-grams: an association measures study. International Review of Applied Linguistics, 58(1), 51-74. doi: KoreaHigh school graduates applying to attend universityStudents taking Yonsei University English placement testNot specified450 English placement test essays each from A2, B1, B2 CEFR proficiency levels randomly sampled from Yonsei English Learner Corpus (sub sample totals 1,350 texts). The writing proficiency test requires students to complete 3 writing tasks in 60 minutes:Task 1: Sentence completion from words displayed on a screen.Task 2: Short composition (100 words suggested word limit) on a familiar topic (e.g. your favourite extracurricular activity) and Task 3: Academic essay on a more academic topic (e.g. ‘Why should people receive a college education? State your opinion’) with a suggested word count of 300 words. The analysis is based on task 3 texts only.Opinion-based essayRated by trained native English speaker raters at Yonsei University according to CEFR. Corpus contains 7 proficiency levels from A2-C1 with A1 and C2 texts excluded for their rare occurrence. No inter-rater reliability provided.Garner, Crossley & Kyle 2019Garner, J., Crossley, S. A., & Kyle, K. (2019). N-gram measures and L2 writing proficiency. System, 80, 176-187.South KoreaHigh school graduates applying to attend universityStudents taking Yonsei University English placement testNot specified3,027 essays from A1+ to C1 CEFR proficiency levels. The writing proficiency test requires students to complete 3 writing tasks in 60 minutes:Task 1: Sentence completion from words displayed on a screen.Task 2: Short composition (100 words suggested word limit) on a familiar topic (e.g. your favourite extracurricular activity) and Task 3: Academic essay on a more academic topic (e.g. ‘Why should people receive a college education? State your opinion’) with a suggested word count of 300 words. The analysis is based on task 3 texts only ‘300-word ‘academic’ texts (e.g. ‘Why should people receive a college education? State your opinion’).Opinion-based essayRated by trained native English speaker raters at Yonsei University according to CEFR. Corpus contains 7 proficiency levels from A2-C1 with A1 and C2 texts excluded for their rare occurrence. No inter-rater reliability provided.Gebril & Plakans2009Gebril, A., & Plakans, L. (2009). Investigating source use, discourse features, and process in integrated writing tests. SPAAN FELLOW Working Papers in Second or Foreign Language Assessment, 7, 47-84.UAEUniversity College of Humanities and Social ScienceUndergraduate students Not specified130 essays on global warmingArgumentative integrated essaysIELTS holistic scale. Two raters achieved inter-rater reliability r=.75.Gene-Gil, Juan-Garau & Salazar-Noguera 2015Gené-Gil, G., Juan-Garau, M., & Salazer-Noguera, J. (2015). Development of EFL writing over 3 years in secondary education: CLIL and non-CLIL settings. The Language Learning Journal, 43(3), 286-303.SpainSecondary school30 x CLIL students20 x non-CLIL studentsLongitudinal study: mean 13 years at start; 16 years at endEach student writes 4 x timed compositions. Sampled at: start of grade 8; end of grade 8; end of grade 9; end of grade 10. 25 minute time limitEmailsTopic: Email a friend telling him/her about a film you saw last week. Explain what you did after the film.Longitudinal studyGranger & Bestgen2014Granger, S., & Bestgen, Y. (2014). The use of collocations by intermediate vs. advanced non-ntive writers: A bigram-based study. International Review of Applied Linguistics, 52(3), 229-252.Not specifiedNot specified74 French, 71 German and 78 Spanish students.Not specified223 texts from the ICLE corpus of 500–900 words. Argumentative essays11-point numerical scale that involved allocating + and – grades to CEFR levels B1-C2. Inter-rater reliability r=.69. A holistic score was allocated but also individual scores for vocabulary accuracy, grammatical accuracy, orthographic control,vocabulary range and coherence/cohesionGrant & Ginther 2000Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123-145.Not specifiedProficiency test (Test of Written English)90 studentsNot specified90 x Test of Written English essays. 30 minute time limit. Takes stratified random sample: 30 texts each from scores of 3, 4 and 5. Topic: Preferred source of news and why. Argumentative essaysGraded on holistic 6-point scale by two raters.Green 2012Green, C. (2012). A computational investigation of cohesion and lexical network density in L2 writing. English Language Teaching, 5(8), 57-69. VariousMultiple: university; private language schoolUniversity/language school students with mixed L1s.ICLE students had an average age of 22 years.LOCNESS: 18-21 yearsIndonesian corpora: 14-17 years129 essays in total taken from learner corporaICLE (International Corpus of Learner English): 50 LOCNESS (Louvain Corpus of Native English Essays):45 2 corpora collected from separate EFL schools in Indonesia: 34 ICLE and LOCNESS essays ranged from 250-600 words while essays in the Indonesian corpora contained texts from 250-600 mon topics appearing in all corpora: the importance of environmental conservation and the pros and cons of nuclear power. Untimed essays.Persuasive essays. ICLE data (from university and advanced writers) taken to present ‘higher proficiency’, Indonesian data (from pre-univerity and high school writers) taken to represent ‘lower proficiency’. Random sample of 22% of texts from each corpus independently rated by two raters as ‘high’ or ‘low’ proficiency. All ICLE text assigned to high group, all Indonesian texts to low group, with 100% agreement between ratersGregori-Signes & Clave-Arroitia2015Gregori-Signes, C., & Clavel-Arroiti, B. (2015). Analysing lexical density and lexical diversity in university students' written discourse. Procedia Social and Behavioral Sciences, 198, 546-556.SpainUniversity proficiency test preparation programme54 x 1st year English Studies students at Spanish University, currently taking preparation course for Preliminary English Test (PET)27 x 4th year English Studies students at Spanish UniversityNot specified1st year participants write one text during second week of course and another six weeks later. 150-200 words each. 108 texts.4th year participants write one text each. 27 texts1st year studentsTask 1: Letter to a friendTask 3: Personal statement for an academic application4th year students:Formal letter1st year students assumed to be at CEFR B1; 4th year students at (or approaching) CEFR C2.Guo, Crossley & McNamara2013Guo, L., Crossley, S. A., & McNamara, D. (2013). Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing, 18, 218-238.Not specifiedProficiency test (TOEFL)240 x TOEFL iBT test takersNot specified480 essays: 2 essays per participantIntegrated and independent writing tasks from TOEFL iBTIndependently rated by two trained raters on 5-point scale using TOEFL rubrics. Third rater used for large discrepancies.Gyllstad, Granfeldt, Bernardini & Kallkvist2014Gyllstad, H., Granfeldt, J., Bernardini, P., & K?llkvist, M. (2014). Linguistic correlates to communicative proficiency levels of the CEFR: The case of syntactic complexity in written L2 English, L3 French and L4 Italian. EURASLA Yearbook, 14, 1-30.SwedenHigh school54 pupils across grades 4,9 and final year.10-19 years Time limit of 40 minutes. Task 1: Email to a teacher. Task 2 required students to write a story about something nice or exciting that had happened to them which should have been narrative in style. Students had no access to dictionaries. Narrative; email to teacherDivided into CEFR levels by 8 raters. No mention of inter-rater reliablity checks. CEFR grades ranged from A1-C2.Hirano 1991Hirano, K. (1991). The effect of audience on the efficacy of objective measures of EFL proficiency in Japanese university students. Annual Review of English Language Education in Japan, 2, 21-30.JapanUniversity 158 students: 95 non-English majors and 63 English majors. Students divided into 3 proficiency levelsNot specified30-minute essay on a single given topic. Topic: Would they prefer to live in a large city or in a small town. Essay length not dictated to students. Argumentative essaysProficiency determined by local Comprehensive English Language Test scores.Ho-Peng 1983Ho-Peng, L. (1983). Using t-unit measures to assess writing proficiency of university ESL students. RELC Journal, 14(2), 35-43USAUniversity (UG)60 students randomly selected across 3 proficiency levels (as defined by placement exam). Not specifiedTask 1: rewriting aluminium passage; Task 2: essay on one given topic. Aluminium passage prompt: "Read the passage below all the way through. You will notice that the sentences are short. Study the passage and then rewrite it in a better way. You may combine sentences, change the order of words and omit words that are repeated too many times. Try not to leave out any of the information" (p. 36).Task 2 prompt: "Some people refuse to watch TV because, they argue, it is a complete waste of time...What do you think about these people's viewpoint?" Genre not specified for aluminium passage task. The free essay was designed to elicit and argumentative/persuasive essay.Proficiency based on placement test. Students grouped into those exempt from ESL courses; those required to take one semester of ESL courses; those required to take two semesters of ESL courses.Homburg 1984Homburg, T. J. (1984). Holistic evaluation of ESL compositions: can it be validated objectively? TESOL Quarterly, 18(1), 87-107. USAProficiency text (Michigan Test of English Language Proficiency) 30 students across 3 proficiency levels. Random sample from 3512 students who completed the Michigan Test of English Language Proficiency test to gain university entry.Not specified.30-minute essay on a choice of two topics. Sample topics used: 1) What is the funniest thing that ever happened to you? Describe.2) How should students be evaluated? According to their achievement ortheir effort? Discuss. Students were instructed to write at least 150 words. Not specified.Essays graded on a 10-point scale although only grades 5,6 and 7 are used in analysis.Horst & Collins 2006Horst, M., & Collins, L. (2006). From faible to strong: how does their vocabulary grow? The Canadian Modern Language Review, 63(1), 83-106.CanadaPrimary school (9 x classes)210 L1 French students at beginner proficiency level11-12 yearsTasks: respond to a picture: what had happened before the picture, what was being depicted at that moment, what might happen next? Max time: 20 minutes.Texts elicited from each learner after 100, 200, 300 and 400 hours of ESL instruction (approximately 6 months between first and last text.)NarrativeLongitudinal studyHou, Loerts & Verspoor2016Hou, J., Loerts, H., & Verspoor, M. (2016). Chunk use and development in advanced Chinese L2 learners of English. Language Teaching Research, 22(2), 148-168. ChinaUniversity: CET preparation course18 Chinese students on advanced-level university English programmeNot specifiedFirst and last two texts written by students on 18-month College English Test course. Written in students' own time. 96-399 words.ArgumentativeGeneral proficiency rating based on 5-trait rubric, each trait marked on a 5-point scale. Marked by raters in groups of three. IRR for overall score: .799Hou, Verspoor & Loerts2016Hou, J., Verspoor, M., & Loerts, H. (2016). An exploratory study into the dynamics of Chinese L2 writing development. Dutch Journal of Applied Linguistics, 5(1), 65-96.ChinaHigh school and university 23 x high school students.8 x low proficiency university students.18 x high proficiency university students.High school: 18 years.University group: 20 years. 196 essays: Students wrote 2 essays at the start of their course and 2 at the end of the course (2 years course for high school students; 18 month course for university students)High school group: letters and argumentative.University group: argumentativeEntry test scores decided proficiency levelHsu 2007Hsu, J.-y. (2007). Lexical collocations and their relation to the online writing of Taiwanese college English majors and non-English majors. Electronic Journal of Foreign Language Teaching, 4(2), 192-209. TaiwanUniversity (UG)41 English major students21 Non-English major students participated non-randomly in the study.Not specified45-minute online writing test that produced an essay. Topic was pros and cons on 'Dress for success': Exploring the trend of workplace dress becoming less formal.Not specified.1-6-point holistic scale. Scores allocated automatically by ETS' Criterion writing platform.Huang2015Huang, K. (2015). More does not mean better: frequency and accuracy analysis of lexical bundles in Chinese EFL learners' essay writing. System, 53, 13-23.ChinaUniversity English major (UG)EFL majorsNot specifiedTimed essays:3,954 x Year 1/25,990 x Year 3/4Argumentative Cross-sectional development study.Intaraprawat & Steffensen 1995Intaraprawat, P., & Steffensen, M. S. (1995). The use of metadiscourse in good and poor ESL essays. Journal of Second Language Writing, 4(3), 253-272. USAUniversity (UG & PG) 12 students: 4 x upper-level undergraduate 8 x 1st year postgraduate Not specified47 essays from university placement test. Topic: opposing or supporting the university's English requirements. Persuasive TOEFL scores distinguish between good and poor essays. Good essays had a TOEFL score of 593 while poor essays had a score of 513. 5 raters used a 5 point holistic rubric to grade essays.Ishikawa 1995Ishikawa, S. (1995). Objective measurement of low-proficiency EFL narrative writing. Journal of Second Language Writing, 4(1), 51-69. JapanUniversity foreign language course (UG)57 students: whole class used:Class 1: 28 studentsClass 2: 29 students Not specified2 x 30 min picture descriptions. Three months between tasks. Students write story based on 10 picture frames. NarrativeLongitudinal: tasks were completed 3 months apart. Each class works on different forms of writing homework between tasks.Jafarpur1991Jafarpur, A. (1991). Cohesiveness as a basis for evaluating compositions. System, 19(4), 459-465. IranUniversity Department of Foreign Languages and LinguisticsFreshmen, seniors and graduate students.Not specified38 essays at 3 proficiency levels:9 x ‘advanced’ level essays (defined as those written by MA students)12 x middle level essays (defined as those written by senior English majors)17 x low level essays (defined as those written by iNcoming freshmen)Essay topic: Why did you choose Shiraz UniversityNot specifiedWriting scores determined by using a 1-6 point holistic scale. Essays were rated 2 raters. Inter-rater reliability coefficient = .91Jarvis2002Jarvis, S. (2002). Short-texts, best-fitting curves and new measures of lexical diversity. Language Testing, 19(1), 57-84. FinlandSchool210 school students, grades 5, 7, 11. Native speakers of Finnish or Swedish11-15 yearsEssays based on video prompt.NarrativeGraded on an 24-point holistic scale by two experienced raters. IRR: r=.94Jarvis, Grant, Bikowski & Ferris2003Jarvis, S., Grant, L., Bikowski, D., & Ferris, D. (2003). Exploring multiple profiles of highly rated learner compositions. Journal of Second Language Writing, 12, 377-403. Data set 1: USAData set 2: not specifiedData set 1: University ESL programmeData set 2: proficiency test (TWE)Data set 1: 160 x students on ESL programme at US university. 40 each from L1 Arabic, Chinese, Japanese, Spanish (data from Ferris, 1994)Data set 2: 178 x language learners (data from Grant & Ginther, 2000)Not specifiedData set 1: 160 x ESL placement compositions. Max time allowed: 35 minutes (data from Ferris 1991).Data set 2: 178 TWE essays. Max time allowed: 30 minutes (data from Grant & Ginther 2000)Data set 1: Respond to prompt about culture shockData set 2: either preferred source of news or whether they thought teachers should make learning enjoyable and fun.Data set 1: 10-point holistic scale. Final score is sum of three independent raters.Data set 2: rated by two trained TWE raters using ETS protocol. Final score is mean of two raters.Jin 2015Jin, B. (2015). Exploring the development of lexical verbs in academic writing: a multiple-case study of three Chinese novice researchers. The Asian ESP Journal, 2(1), 7-38. ChinaUniversity3 x male researchers in the Engineering Department of a Chinese UniversityNot specifiedResearch articles written for international journals. 4-5 articles per participant, written across 2-4 years. Total 14 texts.Research Articles (Fluid Physics). Longitudinal studyJohnson 1992Johnson, P. (1992). Cohesion and coherence in compositions in Malay and English. RELC Journal, 23(2), 1-17. MalaysiaHigh school and university 60 students:20 x native speakers of American English.20 x native secondary school Malay speakers 20 x university ESL learners with L1 MalayIn each case, teachers select 10 ‘good’ and 10 ’weak’ texts.Not specified60 essays were submitted across the 3 learner groups:20 x essays halved into ‘good’ and ‘weak’ for native speakers of American English.20 x essays halved into ‘good’ and ‘weak’ for native speakers of Malay writing in Malay.20 x essays halved into ‘good’ and ‘weak’ for ESL learners.Essays were written on different topics and for timed examination purposes. The university texts for ESL students were written for college composition course entry. Students had TOEFL 500-525.Not specifiedFor each student group, teachers selected 10 ‘good’ and 10 ’weak’ texts.Kameen 1979Kameen, P. T. (1979). Syntactic skill and ESL writing quality. In On TESOL '79: The learner in focus (pp. 343-350). Washington DC: TESOL.USAProficiency test (Michigan)50 students:25 x poor writers 25 x good writers Not specified 30-minute essays on various topics. Essays written for the Michigan Test of English Proficiency. Sample topic: Who do you consider the most important person who has ever lived?VariousHolistic scale 0-100. Essays rated by two experienced raters. Good writers scored above 79. Poor writers scored between 68-71.Kawata1992Kawata, K. (1992). Evaluation of free English composition. CASELE Research Bulletin, 22, 49-53.JapanHigh school2nd grade students.Not specifiedStudents are given 50 minutes to write on the topic of ‘Home life’. 44 texts selected for analysis, 11 at each of 4 proficiency levelsNot specifiedStudents placed in groups based on proficiency grade from their previous year of schoolingKim 2014Kim, J.-Y. (2014). Predicting L2 writing proficiency using linguistic complexity measures: A corpus-based study. English Teaching, 69(4), 27-51.South KoreaUniversity (UG)University freshmenNot specified234 essays from Yonsei English Learner Corpus. 39 argumentative texts longer than 100 words randomly sampled at each of 6 proficiency levels.Argumentative Graded against CEFR by trained raters. For analysis, 6 proficiency levels collapsed into 3.Kim & Crossley 2018Kim, M., & Crossley, S. A. (2018). Modelling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing. Assessing Writing, 37, 39-56.Not specifiedVarious480 TOEFL test takers from ESL and EFL countriesNot specified480 source-based and 480 independent essays. Source-based essays took 20 minutes and independent essays took 30 minutes to complete. Source-based essay prompts: effects of fish farming and bird migration. Independent essays: choosing study subjects and the importance of cooperation in today's society. Integrated essays; independent essaysEssays were rated on a 5 point holistic scale by 2 raters. If scores differed by 1 point or less then scores were averaged. If scores differed by 1 point or more, a 3rd rater was used. Kim, Crossley & Kyle 2018Kim, M., Crossley, S. A., & Kyle, K. (2018). Lexical sophistication as a multidimensional phenomenon: relations to second language lexical proficiency, development, and writing quality. The Modern Language Journal, 102(1), 120-141.S.Korea; USAVariousHigh school and university studentsNot specifiedData used were essays from 2 corpora and a separate spoken corpus: 3,031 essays from the Yonsei English Learner Corpus. The lexical proficiency corpus from Crossley et al (2011) contained 180 journal entries from L2 learners and 60 journal entries from native speakers. Written corpora consist of argumentative and free writing journal entries. CEFR grading rubric for the YELC corpus. Lexical proficiency corpus includes grades specifically for lexical proficiency.Knoch, Rouhshad, Oon & Storch2015Knoch, U., Rouhshad, A., Oon, S. P., & Storch, N. (2015). What happens to ESL students' writing after three years of study at an English medium university? Journal of Second Language Writing, 28, 39-52.AustraliaUniversity/Proficiency test (Diagnostic English Language Assessment)31 students. A sub-set of the Knoch, Rouhshad & Storch 2014Not specified300-word essays on an unknown topic. Each participant writes two texts. One at the beginning of three-year university programme, the other at the end. Argumentative 6-point holistic scale. No rater information provided. Knoch, Rouhshad & Storch2014Knoch, U., Rouhshad, A., & Storch, N. (2014). Does the writing of undergraduate ESL students develop after one year of study in an English-medium university? Assessing Writing, 21, 1-17. AustraliaUniversity/Proficiency test (Diagnostic English Language Assessment)101 university studentsNot specifiedWriting section of the Diagnostic English Language Assessment proficiency test. 300-word argumentative essay. Maximum time allowed: 30 minutes. One text composed at start of university programme; second composed one year later.ArgumentativeGraded on 6-point scale for Organization, Content and Form. Overall grade is mean of these scores.Kobayashi & Rinnert 2013Kobayashi, H., & Rinnert, C. (2013). L1/L2/L3 writing development: longitudinal case study of a Japanese multicompetent writer. Journal of Second Language Writing, 22, 4-33. JapanUniversity1 Japanese learnerNot specifiedEssays collected over a 2.5-year period (number not specified).ArgumentativeLongitudinal studyKrzeminska- Adamek2016Krzeminska-Adamek, M. (2016). Lexis in writing: Investigating the relationship between lexical richness and the quality of advanced learners' texts. In M. Pawlak (Ed.), Classroom-oriented research: Reconciling theory and practice (pp. 195-197). Switzerland: Springer.PolandUniversity Department of English (UG)65 x 1st-year students in English Department. Level estimated as CEFR B2-C1.No specifiedEach student composes two texts: one at the beginning of the academic year and one at the end. 220-250 words. Topic of different types of communication. 75 minutes allowed.For-and-against type compositionsMarked by two teachers on scale comprising 25 points for organization, content and language control.Kyle & Crossley2016Kyle, K., & Crossley, S. A. (2016). The relationship between lexical sophistication and independent and source-based writing. Journal of Second Language Writing, 34, 12-24.Not specifiedProficiency test (TOEFL)480 x TOEFL iBT test takersNot specifiedTOEFL iBT written texts. 2 texts per participant. 960 texts in total.Integrated and independent writing tasks from TOEFL iBTTOEFL score - 5-point scaleKyle & Crossley 2017Kyle, K., & Crossley, S. A. (2017). Assessing syntactic sophistication in L2 writing: A usage-based approach. Language Testing, 34(4), 513-535.Not specifiedProficiency test (TOEFL)480 x TOEFL iBT test takers from the TOEFL Public Use DatasetNot specified240 essays x 2 independent argumentative essay prompts. Test takers are required to defend an opinion on a real-life topic. Sample topics are the same used in Kyle & Crossley 2016. Test takers had 30 minutes to write essays of at least 300 words in length.Independent argumentative essays5 point holistic scale used by at least two raters to rate each essay.Kyle & Crossley 2018Kyle, K., & Crossley, S. A. (2018). Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. The Modern Language Journal, 102(2), 333-349.Not specifiedProficiency test (TOEFL)Not specifiedNot specified480 essays in total with 240 essays from one prompt and 240 essays from another prompt. Prompt 1: It is more important to choose to study subjects to prepare for a job or career. Use specific reasons and examples to support your answer. Prompt 2: In today's world, the ability to cooperate well with others is far more important than it was in the past. Use specific reasons and examples to support your answer. Students had 30 minutes to write approximately 300 words.Argumentative essaysEssays were rated on a 5 point holistic scale by 2 raters. If scores differed by 1 point or less then scores were averaged. If scores differed by 1 point or more, a 3rd rater was used. Lahuerta-Martinez2018Lahuerta Martínez, A. C. (2018). Analysis of syntactic complexity in secondary education ELF writers at different proficiency levels. Assessing Writing, 35, 1-11.SpainHigh school188 students:98 x year 3 students at lower intermediate proficiency level90 x year 4 students at intermediate level Not specifiedSingle essay topic: Do you think school uniform should be worn at high school? 30-minute time limitOpinion based essay0-5 holistic scale with scores for content, organisation, language use, vocabulary and mchanics. Larsen-Freeman 1978Larsen-Freeman, D. (1978). An ESL Index of Development. TESOL Quarterly, 12(4), 439-448.USAUniversity placement test212 studentsNot specifiedEssay topic was living in a small town or large city. Essays were 200 words in length. Students had 30 minutes to complete the essays. Not specified.Students divided into five groups based on score for whole placement test. No grading information provided. Larsen-Freeman 1983 (study, 1)Larsen-Freeman, D. (1983). Assessing global second language proficiency. In H. W. Seliger & M. H. Long (Eds.), Classroom-oriented research in second language acquisition (pp. 287-304). Rowley, MA: Newbury House.USAUniversity109 studentsNot specifiedRewriting task based on Hunt’s ‘Aluminum’ passage. Students asked to rewrite 32 single-clause sentences.‘Free writing’ task. Topic: Should ESL students receive letter grades in their ESL courses or be evaluated by a pass/not pass system?Not specified.3 proficiency levels but not clear how these are defined.Larsen-Freeman1983 (study, 2)USA4-level ESL class23 studentsNot specified115 essays (5 essays from each student). One essay written every 2 weeks for 10 weeks. No topic info given.Not specified.Longitudinal studyLarsen-Freeman 2006Larsen-Freeman, D. (2006). The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English. Applied Linguistics, 27(4), 590-619.USAUniversity English class for spouses of students5 x Chinese students27-37 yearsWriting task repeated 4 times over a 6-month period. Topic: Write about a past event that writers want to share. Untimed task. NarrativeLongitudinal studyLarsen-Freeman & Strom 1977Larsen-Freeman, D., & Strom, V. (1977). The construction of a second language acquisition index of development. Language Learning, 27(1), 123-134.USAUniversity language placement test48 graduate and undergraduate students across 5 proficiency levelsNot specifiedEssays on one given topic. Essays were written under exam conditions. A minimum length was assigned but not stated in the study.Not specified.Impressionistic scale with 5 categories: poor, fair, average, good, or excellent used by two raters. They achieved inter rater reliability of r=.97.Laufer1994Laufer, B. (1994). The lexical profile of second language writing: does it change over time? RELC Journal, 25(2), 21-33.IsraelUniversity Department of English Language and Literature 48 freshman university students of English lang/lit divided into two groupsNot specifiedgroup 1: 2 essays 14 weeks apart; group 2: 2 essays 28 weeks apart. Class produced essays. No time limit information provided.EssayLongitudinal studyLaufer1998Laufer, B. (1998). The development of passive and active vocabulary in a second language: Same or different? Applied Linguistics, 19(2), 255-271.IsraelHigh school22 x 10th grade 22 x 11th grade 16-17 yearsComposition of 300-400 words. All on topic Should a government be allowed to limit the number of children in families?EssayProgression defined by student year groupsLaufer & Nation1995Laufer, B., & Nation, P. (1995). Vocabulary Size and Use: Lexical Richness in L2 Written Production. Applied Linguistics, 16(3), 307-322.New Zealand; IsraelUniversity EAP programme; University Department of English Language and Literature22 x EAP students in New Zealand20 x 1st semester English lang/lit students in Israel23 x 2nd semester English lang/lit students in IsraelNot specifiedTwo compositions per student, written in the same ics 'of a general nature' dealing with 'controversial issues'.Maximum one hour per text. Required length 300-350 words.EssayProficiency levels defined by student group: EAP students lowest; 2nd semester lang/lit students highestLaufer & Waldman2011Laufer, B., & Waldman, T. (2011). Verb-noun collocations in second language writing: a corpus analysis of learners' English. Language Learning, 61(2), 647-672.IsraelHigh school & University200 x 'basic' (grades 9 & 10)252 x 'intermediate' (grades 11 & 12)307 x 'advanced' (college & university students)Not specified759 argumentative and descriptive, nontechnical essays.Argumentative and descriptive3 proficiency levels: School/university year was initially used to split groups into basic, intermediate and advanced levels with the college/university learners deemed advanced. Sample of 45 essays from each group graded to confirm reliability of levels. All but one remains in original levelLee 1992Lee, H.-J. (1992). Measures of quality in L1 and L2 writing by two ESL learners. English Teaching, 44, 89-113.USAPrimary school2 Korean children Ages 7-97 text samples in L2 English and 7 text samples in L1 Korean were collectedPersonal journal writings 1-6 point holsitic grading scaleLemmouh2008Lemmouh, Z. (2008). The relationship between grades and the lexical richness of student essays. Nordic Journal of English Studies, 7(3), 163-180.SwedenUniversity English language and literature course (UG)37 'advanced' learners: first year UG students of English lang/litNot specifiedEssays written as part of degree programme requirements.EssayGrades awarded to coursework essaysLevitzky-Aviad 2012Levitzky-Aviad, T. (2012). Lexical richness and variation in the writing of school-age EFL learners at different learning stages and different educational systems. In Y. Tono, Y. Kawaguchi, & M. Minegishi (Eds.), Developmental and crosslinguistic perspectives in learner corpus research (pp. 159-168). Amsterdam/Philadelphia: John Benjamins Publishing Company.Austria, China, Hong Kong, Israel, Japan, Poland, Spain, Taiwan.Primary school & high schoolNot specified Grades 5-12520 texts from International Corpus of Crosslinguistic Interlanguage (ICCI), which contains multiple texts from institutions in 6 different countries. Various topics. Argumentative and descriptive essaysCross-sectional development studyLevitzky-Aviad & Laufer 2013Levitzky-Aviad, T., & Laufer, B. (2013). Lexical properties in the writing of foreign language learners over 8 years of study: Single words and collocations. In C. Bardel, C. Lindquist, & B. Laufer (Eds.), L2 vocabulary acquisition, knowledge and use: New perspectives on assessment and corpus analysis. (pp. 127-148): European Second Language Association.IsraelPrimary school & high school & university215 x school students (grades 6-12)75 x university studentsNot specified215 essays from school children75 essays from university students.Essay topics differ with school year group. 200-word samples taken from each text. Texts shorter than 200 words excluded. Topics for grades 5 to 8: narratives or descriptions e.g. Describe a family event you attended or tell the story of what's going on in the picture.Grades 9-12 and university learners: descriptive and argumentative: What would you do if you got a huge sum of money for your birthday? Explain your choices.Argumentative, narrative and descriptiveNo grading information. Different school years taken as points of development. Li & Schmitt 2009Li, J., & Schmitt, N. (2009). The acquisition of lexical phrases in academic writing: a longitudinal case study. Journal of Second Language Writing, 18, 85-102.UKUniversity MA ELT programme1 Chinese postgraduate student studying MA ELT29 years8 x postgraduate MA assignments + 1 x final dissertation. Topics included a study of word approach studies and motivationGenre not specified but topics indicated that the text type of critical review was popular.Longitudinal studyLiu & Braine2005Liu, M., & Braine, G. (2005). Cohesive features in argumentative writing produced by Chinese undergraduates. System, 33, 623-636.ChinaUniversity basic writing programme50 first-year students randomly chosen from 96 enrolled in Basic Writing module.Not specified50 essays written at the end of 16-week writing course on the topic: Whether Computer Games Should or Should Not Be Forbidden in Universities. Essays were 150-200 words in length.Argumentative25 band holistic scale for each essay (criteria given in article). 2 raters graded each essay. IRR=.91. Final essay score for each text was calculated based on the mean of two raters. Essays scoring above 23 were deemed high quality and those below 16 were deemed low quality. (overall mean for 50 texts = 20.6)Llanes, Tagrant & Serrano2018Llanes, ?., Tragant, E., & Serrano, R. (2015). Examining the role of learning context and individual differences in gains in L2 writing performance: the case of teenagers on an intensive study-abroad programme. The Language Learning Journal, 46(2), 201-216.UKStudy abroad summer programme for international teenagers 64 Spanish/Catalan teenagers (18 males and 46 females)12-17 years10-minute story writing task from a comic book strip. Students had to write the story based on the images and blanked out speech bubbles. Narrative Writing gains measured by pre and post test design. Lorenzo & Rodriguez 2014Lorenzo, F., & Rodríguez, L. (2014). Onset and expansion of L2 cognitive academic language proficiency in bilingual settings: CALP in CLIL. System, 47, 64-72. Spain4 x secondary schools with CLIL immersion programmes244 9th -12th grade students.13-17-yearsEach participant writes a narrative of an historical event that had been part of their core curriculum. Historical narrativeCross-sectional development.Lu 2011Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers' language development. TESOL Quarterly, 46(1), 36-62. ChinaUniversity English major (UG)9 colleges in China, Years 1-4. Only data from one college used in final analysis18-22 years3,554 essays from the Written English Corpus of Chinese Learners. Essays from 82-1,031 words, mean=395. Initial analysis shows interfering effect of institution on results, so developmental analysis performed only on texts from one institution: 943 textsArgumentative, narrative and expository but only expository essays were collected in one institution.Development measures cross-sectionally by year group. Mazgutova & Kormos 2015Mazgutova, D., & Kormos, J. (2015). Syntactic and lexical development in an intensive English for Academic Purposes programme. Journal of Second Language Writing, 29, 3-15. UKUniversity preparation programme25 x pre-PG students14 x pre-UG students18-34 yearsArgumentative writing tasks completed in weeks 1 and 4 of 4-week course. 300-400 words each. Maximum time: 45 minutes. Argumentative Longitudinal studyMeara & Bell 2001Meara, P., & Bell, H. (2001). P-Lex: A simple and effective way of describing the lexical characteristics of short L2 texts. Prospect, 16(3), 5-19. UKUniversity EFL summer course49 EFL learners taking part in summer courses. Proficiency ranging from lower intermediate to advancedNot specifiedEach student writes 2 textsDiscursiveTexts not evaluated: participants divided into high vs. low-proficiency groups based on active Vocabulary Levels TestMoreno Espinosa 2005Moreno Espinosa, S. (2005). Can P_Lex accurately measure lexical richness in the written production of young learners of EFL? Porta Linguarum, 4, 7-21.Spain4 x primary schools184 x 4th-grade students10 yearsEach participant writes one essays as part of normal class. Texts 50-311 words each. Write to your host family introducing yourself.LetterAnalytical scales from the ESL Composition ProfileNihalani 1981Nihalani, N. K. (1981). The quest for the L2 index of development. RELC Journal, 12(2), 50-56. IndiaUniversity29 x learners of English at universityNot specifiedCompositions written at home with no time restrictions. No topic information given.Genre not specifiedTwo graders assign compositions to one of four levels: A: more than satisfactoryB: satisfactoryC: borderlineD: unsatisfactoryOhlrogge2009Ohlrogge, A. (2009). Formulaic expressions in intermediate EFL writing assessment. In R. Corrigan, E. A. Moravcsik, H. Ouali, & K. M. Wheatley (Eds.), Formulaic language Volume 2: Acquisition, loss, psychological reality and functional explanations (pp. 375-386). Amsterdam/Philadelphia: John Benjamins Publishing Company.Not specifiedProficiency test (Michigan)170 test takers from 9 different L1 backgrounds13-50 years. Mean 19.5 years170 x texts written as part of Examination for the Certificate of Competency in English. 30 minutes time-limitPersonal letter or personal essayTexts rated by two independent raters. Third rater used to resolve disagreements. Rated on 5-point scale, with descriptors covering areas of content and development, organization and connectin of ideas, linguistic range and control, communicative effectPalfreyman & Kalraki2017Palfreyman, D. M., & Karaki, S. (2017). Lexical sophistication across languages: a preliminary study of undergraduate writing in Arabic (L1) and English (L2). International Journal of Bilingual Education and Bilingualism, DOI: 10.1080/13670050.2017.1326456.UAEUniversity (UG)First semester undergraduate studentsNot specified35 essays collected over 3 points in a single 16-week semester.ArgumentativeNA Paquot2018Paquot, M. (2018). Phraseological competence: A missing component in university entrance language tests? Insights from a study of EFL learners' use of statistical collocations. Language Assessment Quarterly, 15(1), 29-43.BelgiumUniversity linguistics programme (UG & PG)78 students on BA or BA modern language programmes at a Belgian university.19-26 years98 research papers written for university linguistic course - mean length 3,436 words. Taken from VESPA corpus.Research papersCEFR A1- C2 levels. Each essay graded by two raters – three raters were used in cases of disagreement.Paquot2019Paquot, M. (2019). The phraseological dimension in interlanguage complexity research. Second Language Research, 35(1), 121-145.BelgiumUniversity linguistics programme (UG & PG)78 students on BA or BA modern language programmes at a Belgian university.19-26 years98 research papers written for university linguistic course - mean length 3,436 words. Taken from VESPA corpus.Research papersCEFR A1- C2 levels. Each essay graded by two raters – three raters were used in cases of disagreement.Parkinson & Musgrave 2014Parkinson, J., & Musgrave, J. (2014). Development of noun phrase complexity in the writing of English for Academic Purposes students. Journal of English for Academic Purposes, 14, 48-59. UKMultiple: University preparation programme and University21 x EAP students16 x MA studentsNot specified37 coursework essays in total. Mean word count for EAP essays: 653. Mean word count for MA essays: 786. EAP students wrote argumentative essays while MA students wrote different genre pieces that focused on using and responding to literature. No grading information provided. MA group were labelled high proficiency as their IELTS score was at least 6.5Perez- Paredes & Diez-Bedmar2012Pérez-Paredes, P., & Díez-Bedmar, M. B. (2012). The use of intensifying adverbs in learner writing. In Y. Tono, Y. Kawaguchi, & M. Minegishi (Eds.), Developmental and crosslinguistic perspectives in learner corpus research (pp. 105-125). Amsterdam/Philadelphia: John Benjamins Publishing Company.SpainPrimary schoolPrimary school children from grades 5-10 Grades 5-10226 x descriptive essays (Topic: ‘Which is your favourite film? what happens in it?’)273 x argumentative essays (Topic: ‘Imagine you won the lottery, What would you choose to do with the money?’). . Descriptive essays; argumentative essaysCross-sectional development study Perkins 1980Perkins, K. (1980). Using objective methods of attained writing proficiency to discriminate among holistic evaluations. TESOL Quarterly, 14(1), 61-69. USAUniversity preparation programme29 studentsNot specified50 min essay on one of 3 topics:a) Describe features of American life you have found attractive or unattractive. b) Discuss the importance of foreign language training for better international understanding. c) Choose one area or group in your country that has special problems. Describe the problems and try to indicate why they exist.Not specified.3- point holistic scale: pass, pass minus and fail. Essays rated by two raters.Plakans & Gebril2017Plakans, L., & Gebril, A. (2017). Exploring the relationship of organization and connection with scores in integrated writing assessment. Assessing Writing, 31, 98-112. Not specifiedProficiency test (TOEFL)Various students of different L1s and countries.14-51 years480 integrated and independent TOEFL essaysArgumentative and source-based essays1-5 point holistic TOEFL scale. Plakans, Gebril and Bilki 2019Plakans, L., Gebril, A., & Bilki, Z. (2019). Shaping a score: Complexity, accuracy, and fluency in integrated writing performances. Language Testing, 36(2), 161-179. Not specified Proficiency test (TOEFL)Learners from different non-L1 countries.14-51 years480 integrated writing essays from TOEFL iBT based on listening and reading texts.Writing based on listening and reading materials 1-5 point holistic scaleQin & Uccelli 2016Qin, W., & Uccelli, P. (2016). Same language, different functions: A cross-genre analysis of Chinese EFL learners' writing performance. Journal of Second Language Writing, 33, 3-17. ChinaSecondary school100 x secondary school students from grades 6 to 11.11-17 yearsEach participant wrote two texts. Computer-based writing assessment, conducted as part of regular classroom activities. 40 minutes allowed for each text.Narrative and argumentativeRated independently by two scorers using two genre-specific 6-point holistic scales (IRR:argumentative: k=.83; narrative: k=.80). Third rater used where scores are more than one point apart.Ruan2017Ruan, Z. (2017). Lexical bundles in Chinese undergraduate academic writing at an English medium university. RELC Journal, 48(3), 327-340. ChinaUniversity insessional EAP/ESP programme (UG)Students on 2-year EAP university programme Students on English & international business/finance programmeNot specified229 x EAP Year 1 first essays243 x EAP Year 1 final coursework246 x EAP Year 2 final coursework59 x English & international business/finance Year 4 final year project dissertationsNot specifiedLooks at variation across years of university studyRuegg, Fritz & Holland2011Ruegg, R., Fritz, E., & Holland, J. (2011). Rater sensitivity to qualities of lexis in writing. TESOL Quarterly, 45(1), 63-80. JapanUniversity; Proficiency test (Kanda English Proficiency Test)140 freshmen level students with intermediate to advanced proficiencyNot specified140 essays from a Japanese university proficiency test. Single prompt: Keeping in touch with friends is important for everyone. Emailing, texting by cell phone and social networking sites like Mixi are popular but cause problems for society and among friends. People should communicate and keep in touch through other, more personal ways. Give your reaction to the above statement and support your answer with specific reasons and examples.Genre not specified.Lexis score: Rasch adjusted lexis score from analytic grading scale.Proficiency judgement was based on overall language proficiency with none of the students experienced writing in English prior to attending university.Santos, Verspoor & Nerbonne2012Santos, V. D. O., Verspoor, M., & Nerbonne, J. (2012). Identifying important factors in essay grading using machine learning. In International Experiences in Language Testing and Assessment - Selected Papers in Memory of Pavlos Pavlou (pp. 259-309). Netherlands6 x VWO high schools (high achieving schools)481 1st/3rd year high school students enrolled on regular or semi-immersion English programmes 12-15 yearsEach student writes one text. Maximum 200 words.1st year students write about their new school.3rd year students write about their previous vacation.Not specified0-5-point holistic scale. Essays scored by rating teams (8 raters in total). Initial scoring to subset to create criteria. Groups of 4 rate papers, with majority score used. Where no majority score exists, paper referred to other group.Shaw & Liu 1998Shaw, P., & Liu, E. T. K. (1998). What develops in the development of second-language writing. Applied Linguistics, 19(2), 225-254. UKUniversity preparation programme164 pre-sessional EAP studentsNot specified328 essays with one essay collected at the start and end of the 2-3-month course. Max time allowed: 30 minutes. Topics: 'Preserving the Environment' and 'Twizzlehurst'176 x descriptive and explanatory152 x persuasive-situation-problem-response-evaluation Longitudinal study.Shih & Ma 2012Shih, A., & Ma, M. (2012). Profiling EFL learners' writing performance by syntactic complexity: A corpus-based study. In Y. Tono, Y. Kawaguchi, & M. Minegishi (Eds.), Developmental and crosslinguistic perspectives in learner corpus research (pp. 125-138). Amstermad/Philadelphia: John Benjamins Publishing Company.TaiwanHigh school3 x junior public high schools 3 x senior high schoolsNot specified734 essays from the Taiwan sub-corpus of the International Corpus of Crosslinguistic Interlanguage (ICCI). Sample topics included writing about good and bad days, a funny story and school.Argumentative and descriptiveThe CEFR was used to grade most of the textsSingchai & Jaturapitakkul 2016Singchai, P., & Jaturapitakkul, N. (2016). Cohesion in narrative essay writing of EFL secondary students in Thailand. The New English Teacher, 10(2), 89-111. ThailandHigh school30 x grade 11 studentsGrade 1130 essays on the topic "My hero". Each essay was more than 250 words in length. Essays were written in a one-hour session.Narrative essaysGPA grades decided proficiency levels. GPAs ≥3.5 = advanced, 2.6-3.5= intermediate and ≤2.6 = beginner. Staples, Egbert, Biber & McClair2013Staples, S., Egbert, J., Biber, D., & McClair, A. (2013). Formulaic sequences and EAP writing development: Lexical bundles in the TOEFL iBT writing section. Journal of English for Academic Purposes, 12, 214-225. Not specifiedProficiency test (TOEFL).480 x TOEFL iBT test takersNot specifiedTOEFL iBT written texts. 2 texts per participant. Total 960 texts.Integrated and independent writing tasks from TOEFL iBTTOEFL scoreStaples & Reppen 2016Staples, S., & Reppen, R. (2016). Understanding first-year L2 writing: A lexico-grammatical analysis across L1s, genres, and language ratings. Journal of Second Language Writing, 32, 17-35. USAUniversity UG writing courseArabic and Chinese ESL university students and NS university students all studying a first-year undergraduate writing course Not specified120 essays (40 per L1 group).Average word counts: Rhetorical argument = 1034.25 words and long argument = 2374.55 words. Sample rhetorical analysis prompt/instructions: The objective of this assignment is to enable you to write a well-constructed Rhetorical Analysis that makes a claim regarding the rhetorical tools exhibited in an advertisement or other image.In a Rhetorical Analysis, you need to focus on some of the following aspects: Contextualize a piece of rhetoric (author, audience, and purpose) Identify the rhetorical appeals (logos, ethos, pathos) Discuss the structure/layout of the image/advertisement (conventions).All essays were untimed and written outside class. Students had to include 8-10 sources.Rhetorical argument and long argument. 5 raters rated the texts. Inter-rated reliability reported as .88 for language use sub score but organisation below .80.Storch2009Storch, N. (2009). The impact of studying in a second language (L2) medium university on the development of L2 writing. Journal of Second Language Writing, 18(2), 103-118. AustraliaUniversity25 graduate level students who had not accessed English language support(subset of data from Storch & Hill 2008) Not specifiedEach participant writes one essay in week 1 of semester and a second in week 12. Same prompt used on each occasion. Time limit: 45 minutes excluding 5 minutes reading short excerpts and 5 minutes planning. Essays were a minimum of 300 words on the issue of animal rights.Not specified.Each paper assessed independently by two raters. Discrepancies resolved through discussion. Rated on three scales: fluency, content, form. Three scores averaged for overall score.Storch & Tapper 2009Storch, N., & Tapper, J. (2009). The impact of an EAP course on postgraduate writing. Journal of English for Academic Purposes, 8, 207-223. AustraliaUniversity (credit-bearing PG EAP programme)69 graduate level students taking a credit-bearing EAP course.Not specifiedEach student produced two essays. First essay at semester start and second essay on the same topic at the end of the 10-week semester. Descriptive essays on the prompt: Describe an important issue/ concept/ process that is relevant to your field of study. Common topics included: ‘language development in lingua franca’ and ‘preventing AIDS’. Max time allowed: 30 mins.Descriptive essayLongitudinal studyStorch & Hill 2008Storch, N., & Hill, K. (2008). What happens to international students’ English after one semester at university? Australian Review of Applied Linguistics, 31(1), 4.1-4.17.AustraliaUniversity 39 x (mostly) East Asian Postgraduate students. Intermediate to Upper Intermediate level of English.Not specifiedWriting task on a diagnostic English test. No task details given.Longitudinal study. Essays graded according to fluency, content and form. Essays double marked.Struc & Wood 2011Struc, N., & Wood, N. (2010). A corpus-based analysis of Japanese university-level learners' L2 writing development over a one-year period. Reitaku University Journal, 90, 155-184. JapanUniversityJapanese studentsNot specified170 essays collected over two years from 2 prompts: Prompt 1: “Imagine two friends went shopping together lastweek. One friend returned home happy, the other friend returned homesad. Write a story about what happened. Prompt 2: “Studying English abroad. Please write reasons for and against studying English in another country". Argumentative and narrativeLongitudinal studyTaguchi, Crawford, & Wetzel 2013Taguchi, N., Crawford, W. J., & Wetzel, D. (2013). What linguistic features are indicative of writing quality? A case of argumentative essays in a college composition program. TESOL Quarterly, 47, 420-430. USAUniversity language placement testStudents at a private mid-west university. Students had mixed L1s: Chinese, Korean, Hindi, Spanish and German.Approximately 18 years116 essays from placement tests. High rated essays contained a mean word count of 1,097 words. Low rated essays contained a mean word count of 837.Source-based argumentative essays. Jacob et al’s (1981) ESL Composition Scale: 5 category analytical scale: language use (25 points), content (30 points), organisation (20 points), and mechanics (5 points). High and low rated essays were decided on points score:≥ 90 = high rated essays. These students were allowed to join native speaker composition class.≤80 = low rated essays. These students joined a non-native speaker composition class.Tedick 1990Tedick, D. J. (1990). ESL writing assessment: Subject-matter knowledge and its impact on performance. English for Specific Purposes, 9, 123-143.USAUniversity ESL composition programme105 graduate students on university ESL programmeNot specifiedGeneral and field specific essays. Each written in 48-minute class sessionNot specified.3 proficiency groups formed based on level of class, as determined by written placement testTorras and Celaya2001Torras, M. R., & Celaya, M. L. (2001). Age-related differences in the development of written production. An empirical study of EFL school learners. International Journal of English Studies, 1(2), 25-45. SpainSchool63 students; 42 started learning English at 8 years; 21 at 11 years10-14 yearsFor each group, data gathered after 200 and 416 hours of instruction. Participants given 10 minutes to write on the topic 'introduce yourself'Not specified.Longitudinal studyTreffers-Daller, Parslow & Williams2018Treffers-Daller, J., Parslow, P., & Williams, S. (2016). Back to basics: How measures of lexical diversity can help discriminate between CEFR levels. Applied Linguistics, Advanced access. DOI:10.1093/applin/amw009. Not specifiedProficiency test (PTE-A)179 adult learners of English taking Pearson Test of English Academic16-51 yearsWriting part of PTE-A. Texts are 200-300 words. 200 words sampled from each text.Not specified.Each writer assigned to one of four CEFR levels based on complete PTE-A test (not just writing). Vajjala2018Vajjala, S. (2018). Automated assessment of non-native learner essays: Investigating the rols of linguistic features. International Journal of Artificial Intelligence in Education, 28(1), 79-105.Not specifiedProficiency test (TOEFL & FCE)Not specifiedNot specifiedCorpus 1: Texts sampled from 12,1000 essays written in response to TOEFL iBT independent writing task. 1,069 texts sampled for each of three levels (high/medium/low) Corpus 2: First Certificate of English corpusCorpus 1: TOEFL iBT independent writing task Corpus 2: variousCorpus 1: 6 band score. Divided for study into high/medium/low Corpus 2: 1-40 scaleVerspoor, Lowie, Chan & Vahtrick(study 1)2017Verspoor, M., Lowie, W., Chan, H. P., & Vahtrick, L. (2017). Linguistic complexity in second language development: variability and variation at advanced stages. Recherches en Didactique des Langues et des Cultures, 14(1), 1-27. NetherlandsUniversity English major (UG/PG)1 student studying English at a Dutch university 18-22 years22 texts over 4 years: 7 papers per year for 3 years: 10 papers x language, 8 papers x literature and 3 x linguistics. Linguistics papers were part of MA degree. 1 text in year 4. All papers were written at home. 200 - word section was randomly selected for analysis after quotes and reference lists were removed.Various1-5 point holistic scale was used by 6 instructors to rate the texts. Inter-rater reliability was 77%Verspoor, Lowie, Chan and Vahtrick (study 2)2017NetherlandsUniversity English major2 students studying English at a Dutch universityNot specifiedTexts written for various university courses.VariousLongitudinal studyVerspoor, Schmid & Xu2012Verspoor, M., Schmid, M. S., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21(3), 239-263. Netherlands6 x secondary schools. 8 classes from CLIL stream; 14 non-CLIL.489 x students on pre-university programme. Years 1 and 312-15 yearsCompositions on simple personal topics. Year 1 students wrote about their new high school. Year 3 students wrote about a previous holiday. No time limit. 25-200 words.Texts with more Dutch than English words or fewer than 25 words excluded, yielding 437 samples.Narrative essays. Holistic grade on 5-point scale. Criteria determined through extensive norming phase. Graders work in groups of 4, taking majority score and resolving discrepancies through discussion.Vidakovic & Barker2010Vidakovic, I., & Barker, F. (2010). Use of words and multi-word units in Skills for Life Writing examinations. University of Cambridge ESOL Examinations Research Notes, 7-14(41). Not specifiedProficiency tests: Cambridge ESOL Skills for Life testNot specifiedNot specifiedTexts written for Skills for Life examination. 20 texts at each of 5 entry levels (CEFR A1-C1)Various - depending on exam entry level but samples included letters and emails in formal and informal registers.Proficiency based on exam entry levelWang 2014Wang, X. (2014). The relationship between lexical diversity and EFL writing proficiency. University of Sydney Papers in TESOL, 9, 65-88. ChinaHigh school45 senior high school students18-19 yearsEmail written for National Matriculation Test practice test. Approximately 100 words. 45 texts sampled from top three proficiency levels only; 15 texts taken from each level.EmailsGraded by two raters to five proficiency levels based on criteria related to task achievement, use of vocabulary, accuracy of grammar, coherence and cohesion. IRR: r=.98.Xudong, Cheng, Varaprasad & Leng 20102010Xudong, D., Cheng, L. K., Varaprasad, C., & Leng, L. M. (2010). Academic writing development of ESL/EFL graduate students in NUS. Reflections on English Language Teaching, 9(2), 119-138. Not specifiedUniversity English for Academic Purposes course31 PG students studying on intermediate level EAP courseNot specified450-500-word essays from the Diagnostic English Test. Prompts: (a) In the text in Section A, Lee Kwan Yew suggests that Singaporeans who havereceived “an education and opportunities” provided by Singapore cannot leave Singapore permanently with a clear conscience. Do you agree it is wrong to leave your country permanently in search of a better life? Support your answer with ideas and examples from the texts and from your own experience and observation.(b) The texts in sections A and B suggest that governments are making specialefforts to attract foreign talent. Would such a policy be beneficial to yourcountry? Support your answer with ideas and examples from the texts andfrom your own experience and observation.Argumentative essaysHolistic and analytical scoring across an 11 week period. Yang, Lu & Weigle2015Yang, W., Lu, X., & Weigle, S. C. (2015). Different topics, different discourse: relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing, 28, 53-67. USAUniversity (PG: various disciplines)190 graduate students21-46 years380 essays on 2 topics: focus on personal appearance and young people's futures. 30-minute time limitArgumentative essays1-5 TOEFL holistic scale. Yang & Sun2012Yang, W., & Sun, Y. (2012). The use of cohesive devices in argumentative writing by Chinese EFL learners at different proficiency levels. Linguistics and Education, 23, 31-48. ChinaUniversity English major (UG)30 x second year undergraduates30 x fourth year undergraduates All students of English language and literature19-22 years60 essays written under 45-minute time limit. Essays were instructed to be 250 words or more. Argumentative essaysAll papers marked independently by three raters using TOEFL independent writing rubric. Final score is mean of three grades. IRR=.824th year students assumed to represent ‘advanced’ proficiency. 2nd year assumed to represent ‘intermediate’ proficiencyYau 1991Yau, M. S. S. (1991). The role of language factors in second language writing. In L. Malave & G. Duquette (Eds.), Language, culture and cognition: A collection of studies in first and second language acquisition (pp. 266-283). Clevedon: Multilingual Matters.China (Hong Kong)High school60 high school students:20 x grade 9 ESL students from Hong Kong20 x grade 13 ESL students from Hong Kong 20 x grade 9 NS students from CanadaNot specified40-minute essays approximately 200-words in length. Essays based on the topic: What is an ideal person. ESL grade 9 mean word count: 186.8, ESL grade 13 mean word count: 229.3, grade 9 NS mean word count: 363.2. Expository essayBeginner and advanced proficiency level status decided on number of years producing free writing. Beginner level students had just started producing free writing. Advanced level students had approximately 5 years' experience producing free writing. No grading information provided. Yau and Belanger 1984Yau, M. S. S., & Belanger, J. (1984). The influence of mode on the syntactic complexity of EFL students at three grade levels. TESL Canada Journal, 2(1), 65-76. China (Hong Kong)High school60 high school students in forms 3, 5, 7.Not specified.Each student wrote two essays. Time limit not specified Narrative and expositoryGrading information not provided. Proficiency level is operationalised as school year.Yoon2016Yoon, H.-J. (2016). Association strength of verb-noun combinations in experienced NS and less experienced NNS writing: Longitudinal and cross-sectional findings. Journal of Second Language Writing, 34, 42-57.USAUniversity preparation programme51 high-intermediate students. 38 x male; 13 x female. 42 undergraduate; 9 graduate.18-39 years. Mean 22.3 yearsEach participants produced six essays throughout 15-week semester. Three essays for each genre. Essays collected at 2-3 week intervals and handwritten to 30-minute time-limit. Order of topics counterbalanced. Total 306 texts.Argumentative; narrativeLongitudinal studyYoon 2017Yoon, H.-J. (2017). Linguistic complexity in L2 writing revisted: Issues of toic, proficiency, and construct multidimensionality. System, 66, 130-141.Not specifiedUniversityCollege level learners of English from ChinaMean 19 years1,198 essays sampled from International Corpus Network of Asian Learners of English (ICNALE). Each student wrote 2 essays. 40-80 minutes between the 2 essays. Topics: 1. It is important for college students to have a part-time job and 2. Smoking should be completely banned at all the restaurants in the country. 178-483 words.Argumentative Writers classified as CEFR A2 vs. B1.1 vs B1.2 vs. B2 based on proficiency test or vocabulary size test.Yoon & Polio 2016Yoon, H.-J., & Polio, C. (2016). The linguistic development of students of English as a second language in two written genres. TESOL Quarterly, 51(2), 275-301.USAUniversity intensive ESL programme37 adult ESL students at a US university Mean 22 yearsEach participant wrote 3 x narrative and 3 x argumentative texts over a 15-week semester. 2-3 weeks between each text, alternating between genres. Time allowed: 30 minutes. Same prompts used as Yoon (2016)Narrative and argumentative essaysProgression/development judged over 2-3-week essay collection periods. Youn2014Youn, S. J. (2014). Measuring syntactic complexity in L2 pragmatic production: Investigating relationships among pragmatics, grammar, and proficiency. System, 42, 270-287. USAUniversity40 ESL learners at a U.S university studying on preparatory ESL programmesNot specifiedEssays were written on 4 tasks: write a recommendation letter request email to a professor; write an email to a potential employer to send your application packet; write an email to refuse a professor's request of helping with your classmate's class project; write constructive comments on a cover letter written by a classmate. Approximately one hour. LetterParticipants grouped into proficiency levels based on existing TOEFL iBT scores or on the preparatory ESL course they were currently taking.Writing tasks rated for ‘pragmatic performance’ by three raters using task-dependent 3-point analytic scales. Not clear if each text had multiple raters.Yousofi & Bahramlou 2014Yousofi, N., & Bahramlou, K. (2014). Assessing writing quality: vocabulary profiles in place of holistic measures. Iranian EFL Journal, 10(6), 323-344. IranUniversity English Literature and English Translation programme (UG)108 students 18-24 yearsEssay writing took place in a 60-minute session. Topic not specified but was on a generic controversial topic. Students were instructed to write at least 300 words. Argumentative/opinion essaysMichigan Test of English Language Proficiency was used to judge general proficiency level. Essays scored using the ESL Composition Profile.Yu 2010Yu, G. (2010). Lexical Diversity in Writing and Speaking Task Performances. Applied Linguistics, 31(2), 236-259. Not specifiedProficiency test (Michigan)Not specifiedNot specifiedWriting from Michigan English Language Assessment Battery archive. 200 texts selected, covering two personal and three impersonal topics.Not specified.Score assigned from test. Separate analyses for written composition score and overall proficiency score.Zhang2000Zhang, M. (2000). Cohesive features in the expository writing of undergraduates in two Chinese universities. RELC Journal, 31(1), 61-95.ChinaUniversity English major (UG)107 second year students majoring in English at 2 Chinese universities (4 classes).Mean age: 20.5 years & 20.6 years for the two universities 107 essays collected from two universities on the Importance of planting trees and ways of keeping fit. Students had one hour to produce an essay of minimum 250 words.Expository essaysESL Composition Profile analytical scale. 2 raters assessed each essay. Divided into 3 Grades (A, B, C) representing good, average, poor.Zheng2012Zheng, Y. (2012). Exploring long-term productive vocabulary development in an EFL context: The role of motivation. System, 40, 104-119.ChinaUniversity English major (UG)4 x 3rd year students on English majorNot specifiedWriting samples taken from compositions written for compulsory Academic Writing course. 500-1,000 words per essay. Participants required to develop arguments on a debatable issue or critically review a literary work or movie. Texts collected at 3 sessions, with interval of 3-4 months.Argumentative writingLongitudinal studyZheng2016Zheng, Y. (2016). The complex, dynamic development of L2 lexical use: A longitudinal study on Chinese learners of English. System, 56, 40-53ChinaUniversity English major (UG)15 first year students18-19 years 120 essays in total analysis. Each essay between 400-500 words. Topics included: What are your expectations of the working world? What else do you hope to get out of a job besides a regular paycheck?Argumentative essays. Based on university year with first years judged to be intermediate proficiency level. Zyad 2012Zyad, H. (2012). L2 writing development: The case of two high-achieving and two struggling college-level students. International Journal for 21st Century Education, 3(1), 25-36.MoroccoUniversity 2 x high achieving students2 x low achieving studentsNot specifiedOnline assignment postings for ‘paragraph writing’ course. 2 paragraph assignments per participant (first and last assignments for 12-week course): 8 assignments in total.Not specified.Proficiency level decided by researcher based on students' literacy in English. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download