Title: Teacher Assessment of Grammatical Ability in Second ...
Title: Teacher Assessment of Grammatical Ability in Second Language Academic Writing: A Case StudyAuthor: Heike NeumannFaculty of Education, McGill University, 3700 McTavish Street, Montreal, Quebec, Canada. Current/Permanent affiliation: Department of Education, Concordia University, 1455 De Maisonneuve Blvd. West, LB 579, Montreal, Quebec, H3G 1M8, Canada. Tel: 1-514-848-2424 ext. 2443. Email: hneumann@education.concordia.caAbstractIn the language assessment literature, grammatical ability is widely accepted as a key component of second language (L2) ability in general and L2 writing ability in particular. Indicators of grammatical ability have been investigated in L2 writing research, but the indicators L2 writing teachers attend to when determining grammatical ability levels of their students have not been studied. Furthermore, there is no research on what students know about their teachers’ assessment criteria and how that knowledge might affect their writing and learning process. This mixed methods triangulation study examines these questions in university L2 academic writing classes through a quantitative text-based analysis of academic essay exams, student questionnaires, and teacher and student interviews. The combined results of all data sources indicate that the teachers in this study focus primarily on accuracy when assessing grammatical ability. This leads to risk avoidance behaviour by students and may have a negative impact on their learning as students adapt their writing to meet above all their teachers’ expectations for grammatical accuracy. Keywords: Second Language Writing, Academic Writing, Grammatical Ability, Assessment Criteria, Rating ScalesAcknowledgementsThis study was undertaken in partial fulfillment of the requirements of the author’s PhD in Second Language Education at McGill University. Earlier versions of this paper were presented at the Symposium on Second Language Writing in Phoenix, AZ in November 2009 and at the Language Testing Research Colloquium in Cambridge, United Kingdom in April 2010. The author would like to thank Alison Crump, Bonnie Barnett, Kerri Staples, and May Tan for their help in data transcription and analysis, and Carolyn E. Turner, Beverly A. Baker and the anonymous reviewers for their helpful comments on earlier versions of this paper. This study was supported by a doctoral bursary from the FQRSC (Fonds québécois de la recherche sur la société et la culture).IntroductionCumming ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Cumming</Author><Year>2004</Year><RecNum>431</RecNum><DisplayText>(2004)</DisplayText><record><rec-number>431</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">431</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Cumming, Alister</author></authors></contributors><titles><title>Broadening, deepening, and consolidating</title><secondary-title>Language Assessment Quarterly</secondary-title></titles><periodical><full-title>Language Assessment Quarterly</full-title></periodical><pages>5-18</pages><volume>1</volume><number>1</number><dates><year>2004</year></dates><publisher>Routledge</publisher><isbn>1543-4303</isbn><urls><related-urls><url> </electronic-resource-num><access-date>July 08, 2010</access-date></record></Cite></EndNote>(2004) has called for a “broadening” of the investigative scope of writing assessment because “we need to know more about the practices and principles of assessment in ordinary contexts of teaching and learning [italics added]” (p. 6), a call supported by McNamara ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>McNamara</Author><Year>2001</Year><RecNum>481</RecNum><DisplayText>(2001)</DisplayText><record><rec-number>481</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">481</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>McNamara, Timothy F.</author></authors></contributors><titles><title>Rethinking alternative assessment</title><secondary-title>Language Testing</secondary-title></titles><periodical><full-title>Language Testing</full-title></periodical><pages>329-332</pages><volume>18</volume><number>4</number><dates><year>2001</year><pub-dates><date>October 1, 2001</date></pub-dates></dates><urls><related-urls><url>;(2001) and Rea-Dickins ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Rea-Dickins</Author><Year>2009</Year><RecNum>484</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>484</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">484</key></foreign-keys><ref-type name="Book Section">5</ref-type><contributors><authors><author>Rea-Dickins, Pauline</author></authors><secondary-authors><author>Shohamy, Elana</author><author>Hornberger, Nancy H.</author></secondary-authors></contributors><titles><title>Classroom-based assessment</title><secondary-title>Encyclopedia of language and education</secondary-title></titles><pages>257-571</pages><volume>7: Language testing and assessment</volume><edition>2nd</edition><dates><year>2009</year></dates><pub-location>New York</pub-location><publisher>Springer</publisher><urls></urls></record></Cite></EndNote>(2009), yet we still know very little about the classroom assessment of grammar in academic second language (L2) writing classrooms. This is despite the fact that grammatical knowledge is indispensable in accomplishing a task in an L2 ADDIN EN.CITE <EndNote><Cite><Author>Bachman</Author><Year>1996</Year><RecNum>3</RecNum><DisplayText>(Bachman & Palmer, 1996; Weigle, 2002)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">3</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">532</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Bachman, Lyle F.</author><author>Palmer, Adrian. S.</author></authors></contributors><titles><title>Language testing in practice: Designing and developing useful language tests</title></titles><dates><year>1996</year></dates><pub-location>Oxford</pub-location><publisher>Oxford University Press</publisher><urls></urls></record></Cite><Cite><Author>Weigle</Author><Year>2002</Year><RecNum>5</RecNum><record><rec-number>5</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">5</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">534</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Weigle, Sara Cushing</author></authors></contributors><titles><title>Assessing writing</title></titles><dates><year>2002</year></dates><pub-location>Cambridge</pub-location><publisher>Cambridge University Press</publisher><urls></urls></record></Cite></EndNote>(Bachman & Palmer, 1996; Weigle, 2002) and an essential component of writing ability models PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5IYXllczwvQXV0aG9yPjxZZWFyPjE5ODA8L1llYXI+PFJl
Y051bT43ODwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQmFjaG1hbiwgMTk5MDsgQmFjaG1hbiAmYW1w
OyBQYWxtZXIsIDE5OTY7IEdyYWJlICZhbXA7IEthcGxhbiwgMTk5NjsgSGF5ZXMsIDE5OTY7IEhh
eWVzICZhbXA7IEZsb3dlciwgMTk4MDsgU2NhcmRhbWFsaWEgJmFtcDsgQmVyZWl0ZXIsIDE5ODc7
IFdlaWdsZSwgMjAwMik8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+Nzg8L3JlYy1u
dW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2Vh
djBwcDk5cnVzcGF4eHhyZnpmc3ciPjc4PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dS
RGV3cnRxZ2NBQUFwdk1DRSI+NTkyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkJvb2sgU2VjdGlvbiI+NTwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9y
PkhheWVzLCBKb2huIFIuPC9hdXRob3I+PGF1dGhvcj5GbG93ZXIsIExpbmRhIFMuPC9hdXRob3I+
PC9hdXRob3JzPjxzZWNvbmRhcnktYXV0aG9ycz48YXV0aG9yPkdyZWdnLCBMLiBXLjwvYXV0aG9y
PjxhdXRob3I+U3RlaW5iZXJnLCBFLiBSLjwvYXV0aG9yPjwvc2Vjb25kYXJ5LWF1dGhvcnM+PC9j
b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SWRlbnRpZnlpbmcgdGhlIG9yZ2FuaXphdGlvbiBv
ZiB3cml0aW5nIHByb2Nlc3NlczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5Db2duaXRpdmUgcHJv
Y2Vzc2VzIGluIHdyaXRpbmc8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+My0zMDwv
cGFnZXM+PGtleXdvcmRzPjxrZXl3b3JkPkwxIHdyaXRpbmcgcHJvY2Vzczwva2V5d29yZD48a2V5
d29yZD5tb2RlbDwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4xOTgwPC95ZWFyPjwv
ZGF0ZXM+PHB1Yi1sb2NhdGlvbj5IaWxsc2RhbGUsIE5KPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hl
cj5FcmxiYXVtPC9wdWJsaXNoZXI+PHVybHM+PC91cmxzPjxsYW5ndWFnZT5FbmdsaXNoPC9sYW5n
dWFnZT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5TY2FyZGFtYWxpYTwvQXV0aG9yPjxZ
ZWFyPjE5ODc8L1llYXI+PFJlY051bT44MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+ODA8
L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3
ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjgwPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1p
ZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTk0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl
IG5hbWU9IkJvb2sgU2VjdGlvbiI+NTwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48
YXV0aG9yPlNjYXJkYW1hbGlhLCBNLjwvYXV0aG9yPjxhdXRob3I+QmVyZWl0ZXIsIEMuPC9hdXRo
b3I+PC9hdXRob3JzPjxzZWNvbmRhcnktYXV0aG9ycz48YXV0aG9yPlJvc2VuYmVyZywgUzwvYXV0
aG9yPjwvc2Vjb25kYXJ5LWF1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+S25v
d2xlZGdlIHRlbGxpbmcgYW5kIGtub3dsZWRnZSB0cmFuc2Zvcm1pbmcgaW4gd3JpdHRlbiBjb21w
b3NpdGlvbjwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5BZHZhbmNlcyBpbiBhcHBsaWVkIHBzeWNo
b2xpbmd1aXN0aWNzLCBWb2x1bWUgMjogUmVhZGluZywgd3JpdGluZywgYW5kIGxhbmd1YWdlIGxl
YXJuaW5nPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE0Mi0xNzU8L3BhZ2VzPjxk
YXRlcz48eWVhcj4xOTg3PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5DYW1icmlkZ2U8L3B1
Yi1sb2NhdGlvbj48cHVibGlzaGVyPkNhbWJyaWRnZSBVbml2ZXJzaXR5IFByZXNzPC9wdWJsaXNo
ZXI+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkhheWVzPC9BdXRo
b3I+PFllYXI+MTk5NjwvWWVhcj48UmVjTnVtPjcyPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJl
cj43MjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYw
ZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NzI8L2tleT48a2V5IGFwcD0iRU5XZWIi
IGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41ODc8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVm
LXR5cGUgbmFtZT0iQm9vayBTZWN0aW9uIj41PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo
b3JzPjxhdXRob3I+Sm9obiBSLiBIYXllczwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1
dGhvcnM+PGF1dGhvcj5MZXZ5LCBDLiBNaWNoYWVsPC9hdXRob3I+PGF1dGhvcj5SYW5kc2RlbGws
IFNhcmFoPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5BIG5ldyBmcmFtZXdvcmsgZm9yIHVuZGVyc3RhbmRpbmcgY29nbml0aW9uIGFuZCBh
ZmZlY3QgaW4gd3JpdGluZzwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5UaGUgc2NpZW5jZSBvZiB3
cml0aW5nOiBUaGVvcmllcywgbWV0aG9kcywgaW5kaXZpZHVhbCBkaWZmZXJlbmNlcywgYW5kIGFw
cGxpY2F0aW9uczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4xLTI3PC9wYWdlcz48
bnVtLXZvbHM+MTwvbnVtLXZvbHM+PGtleXdvcmRzPjxrZXl3b3JkPlJldmlzZWQgSGF5ZXMgJmFt
cDsgRmxvd2VyIG1vZGVsPC9rZXl3b3JkPjxrZXl3b3JkPkFmZmVjdCAmYW1wOyBtb3RpdmF0aW9u
IGluIHdyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+bW9kZWwgb2YgcmV2aXNpb24gcHJvY2Vzczwv
a2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4xOTk2PC95ZWFyPjwvZGF0ZXM+PHB1Yi1s
b2NhdGlvbj5NYWh3YWgsIE5KPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5MYXdyZW5jZSBFcmxi
YXVtPC9wdWJsaXNoZXI+PHVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT5FRCA0
NDEgMjQ5PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48cmVtb3RlLWRhdGFiYXNlLW5hbWU+RVJJ
QzwvcmVtb3RlLWRhdGFiYXNlLW5hbWU+PGxhbmd1YWdlPkVuZ2xpc2g8L2xhbmd1YWdlPjwvcmVj
b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkJhY2htYW48L0F1dGhvcj48WWVhcj4xOTkwPC9ZZWFy
PjxSZWNOdW0+MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNw
YXh4eHJmemZzdyI+Mjwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFB
cHZNQ0UiPjUzMTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9y
ZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+QmFjaG1hbiwgTHlsZSBGLjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5GdW5kYW1lbnRh
bCBjb25zaWRlcmF0aW9ucyBpbiBsYW5ndWFnZSB0ZXN0aW5nLjwvdGl0bGU+PC90aXRsZXM+PGRh
dGVzPjx5ZWFyPjE5OTA8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPk94Zm9yZDwvcHViLWxv
Y2F0aW9uPjxwdWJsaXNoZXI+T3hmb3JkIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJs
cz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+QmFjaG1hbjwvQXV0aG9yPjxZ
ZWFyPjE5OTY8L1llYXI+PFJlY051bT4zPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zPC9y
ZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2Vm
NXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4zPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0i
U2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTMyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5h
bWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5CYWNo
bWFuLCBMeWxlIEYuPC9hdXRob3I+PGF1dGhvcj5QYWxtZXIsIEFkcmlhbi4gUy48L2F1dGhvcj48
L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+TGFuZ3VhZ2UgdGVzdGluZyBp
biBwcmFjdGljZTogRGVzaWduaW5nIGFuZCBkZXZlbG9waW5nIHVzZWZ1bCBsYW5ndWFnZSB0ZXN0
czwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjE5OTY8L3llYXI+PC9kYXRlcz48cHViLWxv
Y2F0aW9uPk94Zm9yZDwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+T3hmb3JkIFVuaXZlcnNpdHkg
UHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+V2VpZ2xlPC9BdXRob3I+PFllYXI+MjAwMjwvWWVhcj48UmVjTnVtPjU8L1JlY051bT48cmVj
b3JkPjxyZWMtbnVtYmVyPjU8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4i
IGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjU8L2tleT48a2V5
IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41MzQ8L2tleT48L2ZvcmVp
Z24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48
YXV0aG9ycz48YXV0aG9yPldlaWdsZSwgU2FyYSBDdXNoaW5nPC9hdXRob3I+PC9hdXRob3JzPjwv
Y29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc2Vzc2luZyB3cml0aW5nPC90aXRsZT48L3Rp
dGxlcz48ZGF0ZXM+PHllYXI+MjAwMjwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+Q2FtYnJp
ZGdlPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5DYW1icmlkZ2UgVW5pdmVyc2l0eSBQcmVzczwv
cHVibGlzaGVyPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5HcmFi
ZTwvQXV0aG9yPjxZZWFyPjE5OTY8L1llYXI+PFJlY051bT43MDwvUmVjTnVtPjxyZWNvcmQ+PHJl
Yy1udW1iZXI+NzA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlk
PSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjcwPC9rZXk+PGtleSBhcHA9
IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTg1PC9rZXk+PC9mb3JlaWduLWtl
eXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhv
cnM+PGF1dGhvcj5XaWxsaWFtIEdyYWJlPC9hdXRob3I+PGF1dGhvcj5Sb2JlcnQgQi4gS2FwbGFu
PC9hdXRob3I+PC9hdXRob3JzPjxzZWNvbmRhcnktYXV0aG9ycz48YXV0aG9yPkNocmlzdG9waGVy
IE4uIENhbmRsaW48L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPlRoZW9yeSAmYW1wOyBwcmFjdGljZSBvZiB3cml0aW5nOiBBbiBhcHBsaWVk
IGxpbmd1aXN0aWMgcGVyc3BlY3RpdmU8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBwbGllZCBM
aW5ndWlzdGljcyBhbmQgTGFuZ3VhZ2UgU3R1ZHk8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48
cGFnZXM+NDg3PC9wYWdlcz48a2V5d29yZHM+PGtleXdvcmQ+d3JpdGluZyBwcm9jZXNzLCB0aGVv
cnkgb2Ygd3JpdGluZywgc2Vjb25kIGxhbmd1YWdlLCBjb250cmFzdGl2ZSByaGV0b3JpYzwva2V5
d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4xOTk2PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2Nh
dGlvbj5OZXcgWW9yazwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+QWRkaXNvbiBXZXNsZXkgTG9u
Z21hbjwvcHVibGlzaGVyPjxpc2JuPjAtNTgyLTU1MzgzLTA8L2lzYm4+PGNhbGwtbnVtPlBFMTQw
NCBHNjUgMTk5NjwvY2FsbC1udW0+PHVybHM+PC91cmxzPjxyZXNlYXJjaC1ub3Rlcz5Db250YWlu
cyBrZXkgd3JpdGluZyBtb2RlbCBmb3IgTDIgd3JpdGluZzwvcmVzZWFyY2gtbm90ZXM+PGxhbmd1
YWdlPkVuZ2xpc2g8L2xhbmd1YWdlPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+AG==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5IYXllczwvQXV0aG9yPjxZZWFyPjE5ODA8L1llYXI+PFJl
Y051bT43ODwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQmFjaG1hbiwgMTk5MDsgQmFjaG1hbiAmYW1w
OyBQYWxtZXIsIDE5OTY7IEdyYWJlICZhbXA7IEthcGxhbiwgMTk5NjsgSGF5ZXMsIDE5OTY7IEhh
eWVzICZhbXA7IEZsb3dlciwgMTk4MDsgU2NhcmRhbWFsaWEgJmFtcDsgQmVyZWl0ZXIsIDE5ODc7
IFdlaWdsZSwgMjAwMik8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+Nzg8L3JlYy1u
dW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2Vh
djBwcDk5cnVzcGF4eHhyZnpmc3ciPjc4PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dS
RGV3cnRxZ2NBQUFwdk1DRSI+NTkyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkJvb2sgU2VjdGlvbiI+NTwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9y
PkhheWVzLCBKb2huIFIuPC9hdXRob3I+PGF1dGhvcj5GbG93ZXIsIExpbmRhIFMuPC9hdXRob3I+
PC9hdXRob3JzPjxzZWNvbmRhcnktYXV0aG9ycz48YXV0aG9yPkdyZWdnLCBMLiBXLjwvYXV0aG9y
PjxhdXRob3I+U3RlaW5iZXJnLCBFLiBSLjwvYXV0aG9yPjwvc2Vjb25kYXJ5LWF1dGhvcnM+PC9j
b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SWRlbnRpZnlpbmcgdGhlIG9yZ2FuaXphdGlvbiBv
ZiB3cml0aW5nIHByb2Nlc3NlczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5Db2duaXRpdmUgcHJv
Y2Vzc2VzIGluIHdyaXRpbmc8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+My0zMDwv
cGFnZXM+PGtleXdvcmRzPjxrZXl3b3JkPkwxIHdyaXRpbmcgcHJvY2Vzczwva2V5d29yZD48a2V5
d29yZD5tb2RlbDwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4xOTgwPC95ZWFyPjwv
ZGF0ZXM+PHB1Yi1sb2NhdGlvbj5IaWxsc2RhbGUsIE5KPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hl
cj5FcmxiYXVtPC9wdWJsaXNoZXI+PHVybHM+PC91cmxzPjxsYW5ndWFnZT5FbmdsaXNoPC9sYW5n
dWFnZT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5TY2FyZGFtYWxpYTwvQXV0aG9yPjxZ
ZWFyPjE5ODc8L1llYXI+PFJlY051bT44MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+ODA8
L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3
ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjgwPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1p
ZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTk0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl
IG5hbWU9IkJvb2sgU2VjdGlvbiI+NTwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48
YXV0aG9yPlNjYXJkYW1hbGlhLCBNLjwvYXV0aG9yPjxhdXRob3I+QmVyZWl0ZXIsIEMuPC9hdXRo
b3I+PC9hdXRob3JzPjxzZWNvbmRhcnktYXV0aG9ycz48YXV0aG9yPlJvc2VuYmVyZywgUzwvYXV0
aG9yPjwvc2Vjb25kYXJ5LWF1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+S25v
d2xlZGdlIHRlbGxpbmcgYW5kIGtub3dsZWRnZSB0cmFuc2Zvcm1pbmcgaW4gd3JpdHRlbiBjb21w
b3NpdGlvbjwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5BZHZhbmNlcyBpbiBhcHBsaWVkIHBzeWNo
b2xpbmd1aXN0aWNzLCBWb2x1bWUgMjogUmVhZGluZywgd3JpdGluZywgYW5kIGxhbmd1YWdlIGxl
YXJuaW5nPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE0Mi0xNzU8L3BhZ2VzPjxk
YXRlcz48eWVhcj4xOTg3PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5DYW1icmlkZ2U8L3B1
Yi1sb2NhdGlvbj48cHVibGlzaGVyPkNhbWJyaWRnZSBVbml2ZXJzaXR5IFByZXNzPC9wdWJsaXNo
ZXI+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkhheWVzPC9BdXRo
b3I+PFllYXI+MTk5NjwvWWVhcj48UmVjTnVtPjcyPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJl
cj43MjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYw
ZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NzI8L2tleT48a2V5IGFwcD0iRU5XZWIi
IGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41ODc8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVm
LXR5cGUgbmFtZT0iQm9vayBTZWN0aW9uIj41PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo
b3JzPjxhdXRob3I+Sm9obiBSLiBIYXllczwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1
dGhvcnM+PGF1dGhvcj5MZXZ5LCBDLiBNaWNoYWVsPC9hdXRob3I+PGF1dGhvcj5SYW5kc2RlbGws
IFNhcmFoPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5BIG5ldyBmcmFtZXdvcmsgZm9yIHVuZGVyc3RhbmRpbmcgY29nbml0aW9uIGFuZCBh
ZmZlY3QgaW4gd3JpdGluZzwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5UaGUgc2NpZW5jZSBvZiB3
cml0aW5nOiBUaGVvcmllcywgbWV0aG9kcywgaW5kaXZpZHVhbCBkaWZmZXJlbmNlcywgYW5kIGFw
cGxpY2F0aW9uczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4xLTI3PC9wYWdlcz48
bnVtLXZvbHM+MTwvbnVtLXZvbHM+PGtleXdvcmRzPjxrZXl3b3JkPlJldmlzZWQgSGF5ZXMgJmFt
cDsgRmxvd2VyIG1vZGVsPC9rZXl3b3JkPjxrZXl3b3JkPkFmZmVjdCAmYW1wOyBtb3RpdmF0aW9u
IGluIHdyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+bW9kZWwgb2YgcmV2aXNpb24gcHJvY2Vzczwv
a2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4xOTk2PC95ZWFyPjwvZGF0ZXM+PHB1Yi1s
b2NhdGlvbj5NYWh3YWgsIE5KPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5MYXdyZW5jZSBFcmxi
YXVtPC9wdWJsaXNoZXI+PHVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT5FRCA0
NDEgMjQ5PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48cmVtb3RlLWRhdGFiYXNlLW5hbWU+RVJJ
QzwvcmVtb3RlLWRhdGFiYXNlLW5hbWU+PGxhbmd1YWdlPkVuZ2xpc2g8L2xhbmd1YWdlPjwvcmVj
b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkJhY2htYW48L0F1dGhvcj48WWVhcj4xOTkwPC9ZZWFy
PjxSZWNOdW0+MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNw
YXh4eHJmemZzdyI+Mjwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFB
cHZNQ0UiPjUzMTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9y
ZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+QmFjaG1hbiwgTHlsZSBGLjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5GdW5kYW1lbnRh
bCBjb25zaWRlcmF0aW9ucyBpbiBsYW5ndWFnZSB0ZXN0aW5nLjwvdGl0bGU+PC90aXRsZXM+PGRh
dGVzPjx5ZWFyPjE5OTA8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPk94Zm9yZDwvcHViLWxv
Y2F0aW9uPjxwdWJsaXNoZXI+T3hmb3JkIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJs
cz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+QmFjaG1hbjwvQXV0aG9yPjxZ
ZWFyPjE5OTY8L1llYXI+PFJlY051bT4zPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zPC9y
ZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2Vm
NXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4zPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0i
U2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTMyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5h
bWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5CYWNo
bWFuLCBMeWxlIEYuPC9hdXRob3I+PGF1dGhvcj5QYWxtZXIsIEFkcmlhbi4gUy48L2F1dGhvcj48
L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+TGFuZ3VhZ2UgdGVzdGluZyBp
biBwcmFjdGljZTogRGVzaWduaW5nIGFuZCBkZXZlbG9waW5nIHVzZWZ1bCBsYW5ndWFnZSB0ZXN0
czwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjE5OTY8L3llYXI+PC9kYXRlcz48cHViLWxv
Y2F0aW9uPk94Zm9yZDwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+T3hmb3JkIFVuaXZlcnNpdHkg
UHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+V2VpZ2xlPC9BdXRob3I+PFllYXI+MjAwMjwvWWVhcj48UmVjTnVtPjU8L1JlY051bT48cmVj
b3JkPjxyZWMtbnVtYmVyPjU8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4i
IGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjU8L2tleT48a2V5
IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41MzQ8L2tleT48L2ZvcmVp
Z24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48
YXV0aG9ycz48YXV0aG9yPldlaWdsZSwgU2FyYSBDdXNoaW5nPC9hdXRob3I+PC9hdXRob3JzPjwv
Y29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc2Vzc2luZyB3cml0aW5nPC90aXRsZT48L3Rp
dGxlcz48ZGF0ZXM+PHllYXI+MjAwMjwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+Q2FtYnJp
ZGdlPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5DYW1icmlkZ2UgVW5pdmVyc2l0eSBQcmVzczwv
cHVibGlzaGVyPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5HcmFi
ZTwvQXV0aG9yPjxZZWFyPjE5OTY8L1llYXI+PFJlY051bT43MDwvUmVjTnVtPjxyZWNvcmQ+PHJl
Yy1udW1iZXI+NzA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlk
PSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjcwPC9rZXk+PGtleSBhcHA9
IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTg1PC9rZXk+PC9mb3JlaWduLWtl
eXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhv
cnM+PGF1dGhvcj5XaWxsaWFtIEdyYWJlPC9hdXRob3I+PGF1dGhvcj5Sb2JlcnQgQi4gS2FwbGFu
PC9hdXRob3I+PC9hdXRob3JzPjxzZWNvbmRhcnktYXV0aG9ycz48YXV0aG9yPkNocmlzdG9waGVy
IE4uIENhbmRsaW48L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPlRoZW9yeSAmYW1wOyBwcmFjdGljZSBvZiB3cml0aW5nOiBBbiBhcHBsaWVk
IGxpbmd1aXN0aWMgcGVyc3BlY3RpdmU8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBwbGllZCBM
aW5ndWlzdGljcyBhbmQgTGFuZ3VhZ2UgU3R1ZHk8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48
cGFnZXM+NDg3PC9wYWdlcz48a2V5d29yZHM+PGtleXdvcmQ+d3JpdGluZyBwcm9jZXNzLCB0aGVv
cnkgb2Ygd3JpdGluZywgc2Vjb25kIGxhbmd1YWdlLCBjb250cmFzdGl2ZSByaGV0b3JpYzwva2V5
d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4xOTk2PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2Nh
dGlvbj5OZXcgWW9yazwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+QWRkaXNvbiBXZXNsZXkgTG9u
Z21hbjwvcHVibGlzaGVyPjxpc2JuPjAtNTgyLTU1MzgzLTA8L2lzYm4+PGNhbGwtbnVtPlBFMTQw
NCBHNjUgMTk5NjwvY2FsbC1udW0+PHVybHM+PC91cmxzPjxyZXNlYXJjaC1ub3Rlcz5Db250YWlu
cyBrZXkgd3JpdGluZyBtb2RlbCBmb3IgTDIgd3JpdGluZzwvcmVzZWFyY2gtbm90ZXM+PGxhbmd1
YWdlPkVuZ2xpc2g8L2xhbmd1YWdlPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+AG==
ADDIN EN.CITE.DATA (Bachman, 1990; Bachman & Palmer, 1996; Grabe & Kaplan, 1996; Hayes, 1996; Hayes & Flower, 1980; Scardamalia & Bereiter, 1987; Weigle, 2002). Much of the L2 writing assessment research is focused on high-stakes testing situations because of the tests’ consequences for test takers and the tests’ importance for all stakeholders involved. On the other hand, classroom L2 writing research that is related to grammar often centers on pedagogical rather than assessment questions, such as how L2 writing teachers can provide effective feedback so that learning can take place PEVuZE5vdGU+PENpdGUgRXhjbHVkZVllYXI9IjEiPjxBdXRob3I+RmVycmlzPC9BdXRob3I+PFll
YXI+MjAwNzwvWWVhcj48UmVjTnVtPjI4MjwvUmVjTnVtPjxQcmVmaXg+c2VlIDwvUHJlZml4PjxT
dWZmaXg+IDIwMDJgLCAyMDAzYCwgMjAwNyBmb3IgYW4gZXhjZWxsZW50IG92ZXJ2aWV3IG9mIHRo
aXMgbGluZSBvZiByZXNlYXJjaDwvU3VmZml4PjxEaXNwbGF5VGV4dD4oc2VlIEZlcnJpcywgMjAw
MiwgMjAwMywgMjAwNyBmb3IgYW4gZXhjZWxsZW50IG92ZXJ2aWV3IG9mIHRoaXMgbGluZSBvZiBy
ZXNlYXJjaCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjgyPC9yZWMtbnVtYmVy
Pjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5
OXJ1c3BheHh4cmZ6ZnN3Ij4yODI8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdy
dHFnY0FBQXB2TUNFIj43ODQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91
cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9y
PkZlcnJpcywgRGFuYSBSLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5QcmVwYXJpbmcgdGVhY2hlcnMgdG8gcmVzcG9uZCB0byBzdHVkZW50IHdyaXRpbmc8
L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBTZWNvbmQgTGFuZ3VhZ2UgV3JpdGlu
Zzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkpvdXJu
YWwgb2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxw
YWdlcz4xNjUtMTkzPC9wYWdlcz48dm9sdW1lPjE2PC92b2x1bWU+PG51bWJlcj4zPC9udW1iZXI+
PGtleXdvcmRzPjxrZXl3b3JkPldyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+RmVlZGJhY2s8L2tl
eXdvcmQ+PGtleXdvcmQ+UmVzcG9uc2U8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+
MjAwNzwveWVhcj48L2RhdGVzPjxpc2JuPjEwNjAtMzc0MzwvaXNibj48dXJscz48cmVsYXRlZC11
cmxzPjx1cmw+aHR0cDovL3d3dy5zY2llbmNlZGlyZWN0LmNvbS9zY2llbmNlL2FydGljbGUvQjZX
NUYtNFIxMUtGQy0xLzIvMGNmMjJmZDY2ZWUxNjFhZmU3MjEwNmFlNGVlNzBlNjQ8L3VybD48L3Jl
bGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwMTYvai5qc2x3
LjIwMDcuMDcuMDAzIDwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxD
aXRlIEV4Y2x1ZGVBdXRoPSIxIiBFeGNsdWRlWWVhcj0iMSI+PEF1dGhvcj5GZXJyaXM8L0F1dGhv
cj48WWVhcj4yMDAzPC9ZZWFyPjxSZWNOdW0+NDEwPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJl
cj40MTA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJm
MGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQxMDwva2V5PjwvZm9yZWlnbi1rZXlz
PjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz
PjxhdXRob3I+RmVycmlzLCBEYW5hIFIuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPlJlc3BvbnNlIHRvIHN0dWRlbnQgd3JpdGluZzogSW1wbGljYXRpb25z
IGZvciBzZWNvbmQgbGFuZ3VhZ2Ugc3R1ZGVudHM8L3RpdGxlPjwvdGl0bGVzPjxudW1iZXI+eGlp
aSwgMTk0IHAuIDo8L251bWJlcj48a2V5d29yZHM+PGtleXdvcmQ+RW5nbGlzaCBsYW5ndWFnZSBT
dHVkeSBhbmQgdGVhY2hpbmcgRm9yZWlnbiBzcGVha2Vycy48L2tleXdvcmQ+PGtleXdvcmQ+RW5n
bGlzaCBsYW5ndWFnZSBSaGV0b3JpYyBTdHVkeSBhbmQgdGVhY2hpbmcuPC9rZXl3b3JkPjxrZXl3
b3JkPlJlcG9ydCB3cml0aW5nIFN0dWR5IGFuZCB0ZWFjaGluZy48L2tleXdvcmQ+PGtleXdvcmQ+
R3JhZGluZyBhbmQgbWFya2luZyAoU3R1ZGVudHMpPC9rZXl3b3JkPjxrZXl3b3JkPkNvbGxlZ2Ug
cHJvc2UgRXZhbHVhdGlvbi48L2tleXdvcmQ+PGtleXdvcmQ+U2Vjb25kIGxhbmd1YWdlIGFjcXVp
c2l0aW9uLjwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDAzPC95ZWFyPjwvZGF0
ZXM+PHB1Yi1sb2NhdGlvbj5NYWh3YWgsIE4uSi48L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPkxh
d3JlbmNlIEVybGJhdW0gQXNzb2NpYXRlczwvcHVibGlzaGVyPjxpc2JuPjA4MDU4MzY1NlggKGhi
ay4gOiBhbGsuIHBhcGVyKSYjeEQ7MDgwNTgzNjU3OCAocGJrLiA6IGFsay4gcGFwZXIpPC9pc2Ju
Pjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGUgRXhjbHVkZUF1dGg9IjEiIEV4Y2x1
ZGVZZWFyPSIxIj48QXV0aG9yPkZlcnJpczwvQXV0aG9yPjxZZWFyPjIwMDI8L1llYXI+PFJlY051
bT40MTE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQxMTwvcmVjLW51bWJlcj48Zm9yZWln
bi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4
eHJmemZzdyI+NDExPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8
L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5GZXJyaXMsIERhbmEgUi48
L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VHJlYXRtZW50
IG9mIGVycm9yIGluIHNlY29uZCBsYW5ndWFnZSBzdHVkZW50IHdyaXRpbmc8L3RpdGxlPjxzZWNv
bmRhcnktdGl0bGU+TWljaGlnYW4gc2VyaWVzIG9uIHRlYWNoaW5nIG11bHRpbGluZ3VhbCB3cml0
ZXJzPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PG51bWJlcj54aWlpLCAxNTIgcC4gOjwvbnVt
YmVyPjxrZXl3b3Jkcz48a2V5d29yZD5FbmdsaXNoIGxhbmd1YWdlIFN0dWR5IGFuZCB0ZWFjaGlu
ZyBGb3JlaWduIHNwZWFrZXJzLjwva2V5d29yZD48a2V5d29yZD5FbmdsaXNoIGxhbmd1YWdlIFJo
ZXRvcmljIFN0dWR5IGFuZCB0ZWFjaGluZy48L2tleXdvcmQ+PGtleXdvcmQ+UmVwb3J0IHdyaXRp
bmcgU3R1ZHkgYW5kIHRlYWNoaW5nLjwva2V5d29yZD48a2V5d29yZD5FbmdsaXNoIGxhbmd1YWdl
IEVycm9ycyBvZiB1c2FnZS48L2tleXdvcmQ+PGtleXdvcmQ+U2Vjb25kIGxhbmd1YWdlIGFjcXVp
c2l0aW9uLjwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDAyPC95ZWFyPjwvZGF0
ZXM+PHB1Yi1sb2NhdGlvbj5Bbm4gQXJib3I8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlVuaXZl
cnNpdHkgb2YgTWljaGlnYW4gUHJlc3M8L3B1Ymxpc2hlcj48aXNibj4wNDcyMDg4MTY1IChjbG90
aCA6IGFjaWQtZnJlZSBwYXBlcik8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48
L0VuZE5vdGU+AG==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGUgRXhjbHVkZVllYXI9IjEiPjxBdXRob3I+RmVycmlzPC9BdXRob3I+PFll
YXI+MjAwNzwvWWVhcj48UmVjTnVtPjI4MjwvUmVjTnVtPjxQcmVmaXg+c2VlIDwvUHJlZml4PjxT
dWZmaXg+IDIwMDJgLCAyMDAzYCwgMjAwNyBmb3IgYW4gZXhjZWxsZW50IG92ZXJ2aWV3IG9mIHRo
aXMgbGluZSBvZiByZXNlYXJjaDwvU3VmZml4PjxEaXNwbGF5VGV4dD4oc2VlIEZlcnJpcywgMjAw
MiwgMjAwMywgMjAwNyBmb3IgYW4gZXhjZWxsZW50IG92ZXJ2aWV3IG9mIHRoaXMgbGluZSBvZiBy
ZXNlYXJjaCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjgyPC9yZWMtbnVtYmVy
Pjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5
OXJ1c3BheHh4cmZ6ZnN3Ij4yODI8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdy
dHFnY0FBQXB2TUNFIj43ODQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91
cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9y
PkZlcnJpcywgRGFuYSBSLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5QcmVwYXJpbmcgdGVhY2hlcnMgdG8gcmVzcG9uZCB0byBzdHVkZW50IHdyaXRpbmc8
L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBTZWNvbmQgTGFuZ3VhZ2UgV3JpdGlu
Zzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkpvdXJu
YWwgb2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxw
YWdlcz4xNjUtMTkzPC9wYWdlcz48dm9sdW1lPjE2PC92b2x1bWU+PG51bWJlcj4zPC9udW1iZXI+
PGtleXdvcmRzPjxrZXl3b3JkPldyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+RmVlZGJhY2s8L2tl
eXdvcmQ+PGtleXdvcmQ+UmVzcG9uc2U8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+
MjAwNzwveWVhcj48L2RhdGVzPjxpc2JuPjEwNjAtMzc0MzwvaXNibj48dXJscz48cmVsYXRlZC11
cmxzPjx1cmw+aHR0cDovL3d3dy5zY2llbmNlZGlyZWN0LmNvbS9zY2llbmNlL2FydGljbGUvQjZX
NUYtNFIxMUtGQy0xLzIvMGNmMjJmZDY2ZWUxNjFhZmU3MjEwNmFlNGVlNzBlNjQ8L3VybD48L3Jl
bGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwMTYvai5qc2x3
LjIwMDcuMDcuMDAzIDwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxD
aXRlIEV4Y2x1ZGVBdXRoPSIxIiBFeGNsdWRlWWVhcj0iMSI+PEF1dGhvcj5GZXJyaXM8L0F1dGhv
cj48WWVhcj4yMDAzPC9ZZWFyPjxSZWNOdW0+NDEwPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJl
cj40MTA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJm
MGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQxMDwva2V5PjwvZm9yZWlnbi1rZXlz
PjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz
PjxhdXRob3I+RmVycmlzLCBEYW5hIFIuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPlJlc3BvbnNlIHRvIHN0dWRlbnQgd3JpdGluZzogSW1wbGljYXRpb25z
IGZvciBzZWNvbmQgbGFuZ3VhZ2Ugc3R1ZGVudHM8L3RpdGxlPjwvdGl0bGVzPjxudW1iZXI+eGlp
aSwgMTk0IHAuIDo8L251bWJlcj48a2V5d29yZHM+PGtleXdvcmQ+RW5nbGlzaCBsYW5ndWFnZSBT
dHVkeSBhbmQgdGVhY2hpbmcgRm9yZWlnbiBzcGVha2Vycy48L2tleXdvcmQ+PGtleXdvcmQ+RW5n
bGlzaCBsYW5ndWFnZSBSaGV0b3JpYyBTdHVkeSBhbmQgdGVhY2hpbmcuPC9rZXl3b3JkPjxrZXl3
b3JkPlJlcG9ydCB3cml0aW5nIFN0dWR5IGFuZCB0ZWFjaGluZy48L2tleXdvcmQ+PGtleXdvcmQ+
R3JhZGluZyBhbmQgbWFya2luZyAoU3R1ZGVudHMpPC9rZXl3b3JkPjxrZXl3b3JkPkNvbGxlZ2Ug
cHJvc2UgRXZhbHVhdGlvbi48L2tleXdvcmQ+PGtleXdvcmQ+U2Vjb25kIGxhbmd1YWdlIGFjcXVp
c2l0aW9uLjwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDAzPC95ZWFyPjwvZGF0
ZXM+PHB1Yi1sb2NhdGlvbj5NYWh3YWgsIE4uSi48L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPkxh
d3JlbmNlIEVybGJhdW0gQXNzb2NpYXRlczwvcHVibGlzaGVyPjxpc2JuPjA4MDU4MzY1NlggKGhi
ay4gOiBhbGsuIHBhcGVyKSYjeEQ7MDgwNTgzNjU3OCAocGJrLiA6IGFsay4gcGFwZXIpPC9pc2Ju
Pjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGUgRXhjbHVkZUF1dGg9IjEiIEV4Y2x1
ZGVZZWFyPSIxIj48QXV0aG9yPkZlcnJpczwvQXV0aG9yPjxZZWFyPjIwMDI8L1llYXI+PFJlY051
bT40MTE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQxMTwvcmVjLW51bWJlcj48Zm9yZWln
bi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4
eHJmemZzdyI+NDExPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8
L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5GZXJyaXMsIERhbmEgUi48
L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VHJlYXRtZW50
IG9mIGVycm9yIGluIHNlY29uZCBsYW5ndWFnZSBzdHVkZW50IHdyaXRpbmc8L3RpdGxlPjxzZWNv
bmRhcnktdGl0bGU+TWljaGlnYW4gc2VyaWVzIG9uIHRlYWNoaW5nIG11bHRpbGluZ3VhbCB3cml0
ZXJzPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PG51bWJlcj54aWlpLCAxNTIgcC4gOjwvbnVt
YmVyPjxrZXl3b3Jkcz48a2V5d29yZD5FbmdsaXNoIGxhbmd1YWdlIFN0dWR5IGFuZCB0ZWFjaGlu
ZyBGb3JlaWduIHNwZWFrZXJzLjwva2V5d29yZD48a2V5d29yZD5FbmdsaXNoIGxhbmd1YWdlIFJo
ZXRvcmljIFN0dWR5IGFuZCB0ZWFjaGluZy48L2tleXdvcmQ+PGtleXdvcmQ+UmVwb3J0IHdyaXRp
bmcgU3R1ZHkgYW5kIHRlYWNoaW5nLjwva2V5d29yZD48a2V5d29yZD5FbmdsaXNoIGxhbmd1YWdl
IEVycm9ycyBvZiB1c2FnZS48L2tleXdvcmQ+PGtleXdvcmQ+U2Vjb25kIGxhbmd1YWdlIGFjcXVp
c2l0aW9uLjwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDAyPC95ZWFyPjwvZGF0
ZXM+PHB1Yi1sb2NhdGlvbj5Bbm4gQXJib3I8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlVuaXZl
cnNpdHkgb2YgTWljaGlnYW4gUHJlc3M8L3B1Ymxpc2hlcj48aXNibj4wNDcyMDg4MTY1IChjbG90
aCA6IGFjaWQtZnJlZSBwYXBlcik8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48
L0VuZE5vdGU+AG==
ADDIN EN.CITE.DATA (see Ferris, 2002, 2003, 2007 for an excellent overview of this line of research) or how collaborative writing contributes to language and writing development PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XaWdnbGVzd29ydGg8L0F1dGhvcj48WWVhcj4yMDA5PC9Z
ZWFyPjxSZWNOdW0+MjY2PC9SZWNOdW0+PERpc3BsYXlUZXh0PihlLmcuLCBTdG9yY2gsIDIwMDI7
IFN0b3JjaCwgMjAwNTsgU3RvcmNoICZhbXA7IFdpZ2dsZXN3b3J0aCwgMjAwNzsgV2lnZ2xlc3dv
cnRoICZhbXA7IFN0b3JjaCwgMjAwOSwgMjAxMik8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1u
dW1iZXI+MjY2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
ZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yNjY8L2tleT48a2V5IGFwcD0i
RU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43Njg8L2tleT48L2ZvcmVpZ24ta2V5
cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1
dG9ycz48YXV0aG9ycz48YXV0aG9yPldpZ2dsZXN3b3J0aCwgR2lsbGlhbjwvYXV0aG9yPjxhdXRo
b3I+U3RvcmNoLCBOZW9teTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5QYWlyIHZlcnN1cyBpbmRpdmlkdWFsIHdyaXRpbmc6IEVmZmVjdHMgb24gZmx1ZW5j
eSwgY29tcGxleGl0eSBhbmQgYWNjdXJhY3k8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+TGFuZ3Vh
Z2UgVGVzdGluZzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRp
dGxlPkxhbmd1YWdlIFRlc3Rpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz40NDUt
NDY2PC9wYWdlcz48dm9sdW1lPjI2PC92b2x1bWU+PG51bWJlcj4zPC9udW1iZXI+PGRhdGVzPjx5
ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5KdWx5IDEsIDIwMDk8L2RhdGU+PC9wdWIt
ZGF0ZXM+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2x0ai5zYWdlcHVi
LmNvbS9jZ2kvY29udGVudC9hYnN0cmFjdC8yNi8zLzQ0NTwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTE3Ny8wMjY1NTMyMjA5MTA0NjcwPC9l
bGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5XaWdn
bGVzd29ydGg8L0F1dGhvcj48WWVhcj4yMDEyPC9ZZWFyPjxSZWNOdW0+NzU8L1JlY051bT48cmVj
b3JkPjxyZWMtbnVtYmVyPjc1PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVO
IiBkYi1pZD0ieDAydGFlejVmMHd6djNlc3pyNjV6MHo4MHBkNXhmOWVhdGZlIj43NTwva2V5Pjwv
Zm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlw
ZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+V2lnZ2xlc3dvcnRoLCBHaWxsaWFuPC9h
dXRob3I+PGF1dGhvcj5TdG9yY2gsIE5lb215PC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0
b3JzPjx0aXRsZXM+PHRpdGxlPldoYXQgcm9sZSBmb3IgY29sbGFib3JhdGlvbiBpbiB3cml0aW5n
IGFuZCB3cml0aW5nIGZlZWRiYWNrPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgb2Yg
U2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9k
aWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNlY29uZCBMYW5ndWFnZSBXcml0aW5nPC9mdWxs
LXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+MzY0LTM3NDwvcGFnZXM+PHZvbHVtZT4yMTwvdm9s
dW1lPjxudW1iZXI+NDwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29yZD5Db2xsYWJvcmF0aXZlIHdy
aXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+RmVlZGJhY2s8L2tleXdvcmQ+PGtleXdvcmQ+UGFpciB3
b3JrPC9rZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMTI8L3llYXI+PHB1Yi1kYXRl
cz48ZGF0ZT4xMi8vPC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+PGlzYm4+MTA2MC0zNzQzPC9p
c2JuPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vd3d3LnNjaWVuY2VkaXJlY3QuY29t
L3NjaWVuY2UvYXJ0aWNsZS9waWkvUzEwNjAzNzQzMTIwMDA3N1g8L3VybD48L3JlbGF0ZWQtdXJs
cz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPmh0dHA6Ly9keC5kb2kub3JnLzEwLjEw
MTYvai5qc2x3LjIwMTIuMDkuMDA1PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48
L0NpdGU+PENpdGU+PEF1dGhvcj5TdG9yY2g8L0F1dGhvcj48WWVhcj4yMDA3PC9ZZWFyPjxSZWNO
dW0+NDg4PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40ODg8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4
eHhyZnpmc3ciPjQ4ODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIFNl
Y3Rpb24iPjU8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5TdG9yY2gs
IE5lb215PC9hdXRob3I+PGF1dGhvcj5XaWdnbGVzd29ydGgsIEdpbGxpYW48L2F1dGhvcj48L2F1
dGhvcnM+PHNlY29uZGFyeS1hdXRob3JzPjxhdXRob3I+R2FyY8OtYSBNYXlvLCBNYXLDrWEgZGVs
IFBpbGFyPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5Xcml0aW5nIHRhc2tzOiBUaGUgZWZmZWN0cyBvZiBjb2xsYWJvcmF0aW9uPC90aXRs
ZT48c2Vjb25kYXJ5LXRpdGxlPkludmVzdGlnYXRpbmcgdGFza3MgaW4gZm9ybWFsIGxhbmd1YWdl
IGxlYXJuaW5nPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE1Ny0xNzc8L3BhZ2Vz
PjxkYXRlcz48eWVhcj4yMDA3PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5DbGV2ZWRvbiwg
RW5nbGFuZDwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+TXVsdGlsaW5ndWFsIE1hdHRlcnM8L3B1
Ymxpc2hlcj48dXJscz48L3VybHM+PHJlc2VhcmNoLW5vdGVzPkdvb2QgaW50cm8gdG8gd2h5IGNv
bGxhYm9yYXRpb24gaW4gTDIgd3JpdGluZyBjbGFzc3Jvb20uIEZvY3VzIG9mIHN0dWR5OiBBcmUg
TFJFcyByZWxhdGVkIHRvIGJldHRlciB0ZXh0cyAobGluZ3Vpc3RpY2FsbHkpPyBJbiBMUkUgYW5h
bHlzaXMsIGZvY3VzIG9uIGNvbGxlY3RpdmUgc2NhZmZvbGRpbmcgYW5kIGxpa2UgRG9uYXRvICgx
OTk0YSkgZmluZCBldmlkZW5jZSBmb3IgaXQuIEluIGFkZGl0aW9uIHRvIExSRXMgYW5kIGRpc2N1
c3Npb24gYWJvdXQgbGFuZ3VhZ2UsIGNvbGxhYm9yYXRpdmUgd3JpdGluZyBhZmZvcmRzIG9wcG9y
dHVuaXR5IHRvIGNvbGxhYm9yYXRlIHdoZW4gZ2VuZXJhdGluZyBpZGVhcy48L3Jlc2VhcmNoLW5v
dGVzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlN0b3JjaDwvQXV0aG9yPjxZZWFyPjIw
MDU8L1llYXI+PFJlY051bT40ODc8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ4NzwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3
ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NDg3PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl
IG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhv
cnM+PGF1dGhvcj5TdG9yY2gsIE5lb215PC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPkNvbGxhYm9yYXRpdmUgd3JpdGluZzogUHJvZHVjdCwgcHJvY2Vzcywg
YW5kIHN0dWRlbnRz4oCZIHJlZmxlY3Rpb25zPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJu
YWwgb2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48
cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNlY29uZCBMYW5ndWFnZSBXcml0aW5n
PC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+MTUzLTE3MzwvcGFnZXM+PHZvbHVtZT4x
NDwvdm9sdW1lPjxudW1iZXI+MzwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29yZD5Db2xsYWJvcmF0
aXZlIHdyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+U2Vjb25kIGxhbmd1YWdlIHdyaXRpbmc8L2tl
eXdvcmQ+PGtleXdvcmQ+TGVhcm5lcnPigJkgYXR0aXR1ZGUgdG8gcGFpciB3b3JrPC9rZXl3b3Jk
Pjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMDU8L3llYXI+PC9kYXRlcz48aXNibj4xMDYwLTM3
NDM8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVj
dC5jb20vc2NpZW5jZS9hcnRpY2xlL3BpaS9TMTA2MDM3NDMwNTAwMDE3MjwvdXJsPjwvcmVsYXRl
ZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTAxNi9qLmpzbHcuMjAw
NS4wNS4wMDI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48
QXV0aG9yPlN0b3JjaDwvQXV0aG9yPjxZZWFyPjIwMDI8L1llYXI+PFJlY051bT40ODU8L1JlY051
bT48UHJlZml4PmUuZy5gLCA8L1ByZWZpeD48cmVjb3JkPjxyZWMtbnVtYmVyPjQ4NTwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2
MHBwOTlydXNwYXh4eHJmemZzdyI+NDg1PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5h
bWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+
PGF1dGhvcj5TdG9yY2gsIE5lb215PC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPlJlbGF0aW9uc2hpcHMgZm9ybWVkIGluIGR5YWRpYyBpbnRlcmFjdGlvbiBh
bmQgb3Bwb3J0dW5pdHkgZm9yIGxlYXJuaW5nPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkludGVy
bmF0aW9uYWwgSm91cm5hbCBvZiBFZHVjYXRpb25hbCBSZXNlYXJjaDwvc2Vjb25kYXJ5LXRpdGxl
PjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkludGVybmF0aW9uYWwgSm91cm5hbCBv
ZiBFZHVjYXRpb25hbCBSZXNlYXJjaDwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjMw
NS0zMjI8L3BhZ2VzPjx2b2x1bWU+Mzc8L3ZvbHVtZT48bnVtYmVyPjPigJM0PC9udW1iZXI+PGtl
eXdvcmRzPjxrZXl3b3JkPkNsYXNzcm9vbS1iYXNlZCByZXNlYXJjaDwva2V5d29yZD48a2V5d29y
ZD5QYWlyIHdvcms8L2tleXdvcmQ+PGtleXdvcmQ+UGF0dGVybnMgb2YgaW50ZXJhY3Rpb248L2tl
eXdvcmQ+PGtleXdvcmQ+U2Vjb25kIGxhbmd1YWdlIGxlYXJuaW5nPC9rZXl3b3JkPjwva2V5d29y
ZHM+PGRhdGVzPjx5ZWFyPjIwMDI8L3llYXI+PC9kYXRlcz48aXNibj4wODgzLTAzNTU8L2lzYm4+
PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20vc2Np
ZW5jZS9hcnRpY2xlL3BpaS9TMDg4MzAzNTUwMzAwMDA3NzwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTAxNi9zMDg4My0wMzU1KDAzKTAwMDA3
LTc8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjxyZXNlYXJjaC1ub3Rlcz5CdWlsZHMgb24gRG9u
YXRvICgxOTg4KS4gRmluZHMgdGhhdCB0aGVyZSBhIGZvdXIgZGlmZmVyZW50IHBhdHRlcm5zIG9m
IGludGVyYWN0aW9uIGluIGFkdWx0IEVTTCBjbGFzc3Jvb20uIEFsbCBmb3VyIHBhdHRlcm5zIGZv
dW5kIGRpc2N1c3NlZCBpbiBkZXRhaWwgaW4gU3RvcmNoICgyMDAyYSkuIEFyZ3VlcyBmb3IgdGhl
IG5lZWQgb2YgYW5hbHl6aW5nIHRoZSBuYXR1cmUgb2YgcGVlciByZWxhdGlvbnNoaXBzLjwvcmVz
ZWFyY2gtbm90ZXM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT5=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XaWdnbGVzd29ydGg8L0F1dGhvcj48WWVhcj4yMDA5PC9Z
ZWFyPjxSZWNOdW0+MjY2PC9SZWNOdW0+PERpc3BsYXlUZXh0PihlLmcuLCBTdG9yY2gsIDIwMDI7
IFN0b3JjaCwgMjAwNTsgU3RvcmNoICZhbXA7IFdpZ2dsZXN3b3J0aCwgMjAwNzsgV2lnZ2xlc3dv
cnRoICZhbXA7IFN0b3JjaCwgMjAwOSwgMjAxMik8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1u
dW1iZXI+MjY2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
ZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yNjY8L2tleT48a2V5IGFwcD0i
RU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43Njg8L2tleT48L2ZvcmVpZ24ta2V5
cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1
dG9ycz48YXV0aG9ycz48YXV0aG9yPldpZ2dsZXN3b3J0aCwgR2lsbGlhbjwvYXV0aG9yPjxhdXRo
b3I+U3RvcmNoLCBOZW9teTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5QYWlyIHZlcnN1cyBpbmRpdmlkdWFsIHdyaXRpbmc6IEVmZmVjdHMgb24gZmx1ZW5j
eSwgY29tcGxleGl0eSBhbmQgYWNjdXJhY3k8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+TGFuZ3Vh
Z2UgVGVzdGluZzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRp
dGxlPkxhbmd1YWdlIFRlc3Rpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz40NDUt
NDY2PC9wYWdlcz48dm9sdW1lPjI2PC92b2x1bWU+PG51bWJlcj4zPC9udW1iZXI+PGRhdGVzPjx5
ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5KdWx5IDEsIDIwMDk8L2RhdGU+PC9wdWIt
ZGF0ZXM+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2x0ai5zYWdlcHVi
LmNvbS9jZ2kvY29udGVudC9hYnN0cmFjdC8yNi8zLzQ0NTwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTE3Ny8wMjY1NTMyMjA5MTA0NjcwPC9l
bGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5XaWdn
bGVzd29ydGg8L0F1dGhvcj48WWVhcj4yMDEyPC9ZZWFyPjxSZWNOdW0+NzU8L1JlY051bT48cmVj
b3JkPjxyZWMtbnVtYmVyPjc1PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVO
IiBkYi1pZD0ieDAydGFlejVmMHd6djNlc3pyNjV6MHo4MHBkNXhmOWVhdGZlIj43NTwva2V5Pjwv
Zm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlw
ZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+V2lnZ2xlc3dvcnRoLCBHaWxsaWFuPC9h
dXRob3I+PGF1dGhvcj5TdG9yY2gsIE5lb215PC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0
b3JzPjx0aXRsZXM+PHRpdGxlPldoYXQgcm9sZSBmb3IgY29sbGFib3JhdGlvbiBpbiB3cml0aW5n
IGFuZCB3cml0aW5nIGZlZWRiYWNrPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgb2Yg
U2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9k
aWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNlY29uZCBMYW5ndWFnZSBXcml0aW5nPC9mdWxs
LXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+MzY0LTM3NDwvcGFnZXM+PHZvbHVtZT4yMTwvdm9s
dW1lPjxudW1iZXI+NDwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29yZD5Db2xsYWJvcmF0aXZlIHdy
aXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+RmVlZGJhY2s8L2tleXdvcmQ+PGtleXdvcmQ+UGFpciB3
b3JrPC9rZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMTI8L3llYXI+PHB1Yi1kYXRl
cz48ZGF0ZT4xMi8vPC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+PGlzYm4+MTA2MC0zNzQzPC9p
c2JuPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vd3d3LnNjaWVuY2VkaXJlY3QuY29t
L3NjaWVuY2UvYXJ0aWNsZS9waWkvUzEwNjAzNzQzMTIwMDA3N1g8L3VybD48L3JlbGF0ZWQtdXJs
cz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPmh0dHA6Ly9keC5kb2kub3JnLzEwLjEw
MTYvai5qc2x3LjIwMTIuMDkuMDA1PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48
L0NpdGU+PENpdGU+PEF1dGhvcj5TdG9yY2g8L0F1dGhvcj48WWVhcj4yMDA3PC9ZZWFyPjxSZWNO
dW0+NDg4PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40ODg8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4
eHhyZnpmc3ciPjQ4ODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIFNl
Y3Rpb24iPjU8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5TdG9yY2gs
IE5lb215PC9hdXRob3I+PGF1dGhvcj5XaWdnbGVzd29ydGgsIEdpbGxpYW48L2F1dGhvcj48L2F1
dGhvcnM+PHNlY29uZGFyeS1hdXRob3JzPjxhdXRob3I+R2FyY8OtYSBNYXlvLCBNYXLDrWEgZGVs
IFBpbGFyPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5Xcml0aW5nIHRhc2tzOiBUaGUgZWZmZWN0cyBvZiBjb2xsYWJvcmF0aW9uPC90aXRs
ZT48c2Vjb25kYXJ5LXRpdGxlPkludmVzdGlnYXRpbmcgdGFza3MgaW4gZm9ybWFsIGxhbmd1YWdl
IGxlYXJuaW5nPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE1Ny0xNzc8L3BhZ2Vz
PjxkYXRlcz48eWVhcj4yMDA3PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5DbGV2ZWRvbiwg
RW5nbGFuZDwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+TXVsdGlsaW5ndWFsIE1hdHRlcnM8L3B1
Ymxpc2hlcj48dXJscz48L3VybHM+PHJlc2VhcmNoLW5vdGVzPkdvb2QgaW50cm8gdG8gd2h5IGNv
bGxhYm9yYXRpb24gaW4gTDIgd3JpdGluZyBjbGFzc3Jvb20uIEZvY3VzIG9mIHN0dWR5OiBBcmUg
TFJFcyByZWxhdGVkIHRvIGJldHRlciB0ZXh0cyAobGluZ3Vpc3RpY2FsbHkpPyBJbiBMUkUgYW5h
bHlzaXMsIGZvY3VzIG9uIGNvbGxlY3RpdmUgc2NhZmZvbGRpbmcgYW5kIGxpa2UgRG9uYXRvICgx
OTk0YSkgZmluZCBldmlkZW5jZSBmb3IgaXQuIEluIGFkZGl0aW9uIHRvIExSRXMgYW5kIGRpc2N1
c3Npb24gYWJvdXQgbGFuZ3VhZ2UsIGNvbGxhYm9yYXRpdmUgd3JpdGluZyBhZmZvcmRzIG9wcG9y
dHVuaXR5IHRvIGNvbGxhYm9yYXRlIHdoZW4gZ2VuZXJhdGluZyBpZGVhcy48L3Jlc2VhcmNoLW5v
dGVzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlN0b3JjaDwvQXV0aG9yPjxZZWFyPjIw
MDU8L1llYXI+PFJlY051bT40ODc8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ4NzwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3
ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NDg3PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl
IG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhv
cnM+PGF1dGhvcj5TdG9yY2gsIE5lb215PC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPkNvbGxhYm9yYXRpdmUgd3JpdGluZzogUHJvZHVjdCwgcHJvY2Vzcywg
YW5kIHN0dWRlbnRz4oCZIHJlZmxlY3Rpb25zPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJu
YWwgb2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48
cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNlY29uZCBMYW5ndWFnZSBXcml0aW5n
PC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+MTUzLTE3MzwvcGFnZXM+PHZvbHVtZT4x
NDwvdm9sdW1lPjxudW1iZXI+MzwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29yZD5Db2xsYWJvcmF0
aXZlIHdyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+U2Vjb25kIGxhbmd1YWdlIHdyaXRpbmc8L2tl
eXdvcmQ+PGtleXdvcmQ+TGVhcm5lcnPigJkgYXR0aXR1ZGUgdG8gcGFpciB3b3JrPC9rZXl3b3Jk
Pjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMDU8L3llYXI+PC9kYXRlcz48aXNibj4xMDYwLTM3
NDM8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVj
dC5jb20vc2NpZW5jZS9hcnRpY2xlL3BpaS9TMTA2MDM3NDMwNTAwMDE3MjwvdXJsPjwvcmVsYXRl
ZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTAxNi9qLmpzbHcuMjAw
NS4wNS4wMDI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48
QXV0aG9yPlN0b3JjaDwvQXV0aG9yPjxZZWFyPjIwMDI8L1llYXI+PFJlY051bT40ODU8L1JlY051
bT48UHJlZml4PmUuZy5gLCA8L1ByZWZpeD48cmVjb3JkPjxyZWMtbnVtYmVyPjQ4NTwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2
MHBwOTlydXNwYXh4eHJmemZzdyI+NDg1PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5h
bWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+
PGF1dGhvcj5TdG9yY2gsIE5lb215PC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPlJlbGF0aW9uc2hpcHMgZm9ybWVkIGluIGR5YWRpYyBpbnRlcmFjdGlvbiBh
bmQgb3Bwb3J0dW5pdHkgZm9yIGxlYXJuaW5nPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkludGVy
bmF0aW9uYWwgSm91cm5hbCBvZiBFZHVjYXRpb25hbCBSZXNlYXJjaDwvc2Vjb25kYXJ5LXRpdGxl
PjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkludGVybmF0aW9uYWwgSm91cm5hbCBv
ZiBFZHVjYXRpb25hbCBSZXNlYXJjaDwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjMw
NS0zMjI8L3BhZ2VzPjx2b2x1bWU+Mzc8L3ZvbHVtZT48bnVtYmVyPjPigJM0PC9udW1iZXI+PGtl
eXdvcmRzPjxrZXl3b3JkPkNsYXNzcm9vbS1iYXNlZCByZXNlYXJjaDwva2V5d29yZD48a2V5d29y
ZD5QYWlyIHdvcms8L2tleXdvcmQ+PGtleXdvcmQ+UGF0dGVybnMgb2YgaW50ZXJhY3Rpb248L2tl
eXdvcmQ+PGtleXdvcmQ+U2Vjb25kIGxhbmd1YWdlIGxlYXJuaW5nPC9rZXl3b3JkPjwva2V5d29y
ZHM+PGRhdGVzPjx5ZWFyPjIwMDI8L3llYXI+PC9kYXRlcz48aXNibj4wODgzLTAzNTU8L2lzYm4+
PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20vc2Np
ZW5jZS9hcnRpY2xlL3BpaS9TMDg4MzAzNTUwMzAwMDA3NzwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTAxNi9zMDg4My0wMzU1KDAzKTAwMDA3
LTc8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjxyZXNlYXJjaC1ub3Rlcz5CdWlsZHMgb24gRG9u
YXRvICgxOTg4KS4gRmluZHMgdGhhdCB0aGVyZSBhIGZvdXIgZGlmZmVyZW50IHBhdHRlcm5zIG9m
IGludGVyYWN0aW9uIGluIGFkdWx0IEVTTCBjbGFzc3Jvb20uIEFsbCBmb3VyIHBhdHRlcm5zIGZv
dW5kIGRpc2N1c3NlZCBpbiBkZXRhaWwgaW4gU3RvcmNoICgyMDAyYSkuIEFyZ3VlcyBmb3IgdGhl
IG5lZWQgb2YgYW5hbHl6aW5nIHRoZSBuYXR1cmUgb2YgcGVlciByZWxhdGlvbnNoaXBzLjwvcmVz
ZWFyY2gtbm90ZXM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT5=
ADDIN EN.CITE.DATA (e.g., Storch, 2002; Storch, 2005; Storch & Wigglesworth, 2007; Wigglesworth & Storch, 2009, 2012). Little research, however, has been conducted on how L2 writing teachers assess grammar in writing classrooms. The current study examines this issue in the context of university L2 academic writing classes.Assessing Grammar in L2 Academic WritingStudents’ writing ability is usually assessed by means of a performance test. Although portfolio assessment has seen a surge in popularity since the 1990s as an alternative to impromptu essay exams and Byrnes and her colleagues have expanded the definition of what writing assessment tasks might look like ADDIN EN.CITE <EndNote><Cite><Author>Byrnes</Author><Year>2010</Year><RecNum>412</RecNum><DisplayText>(Byrnes, 2002; Byrnes, Maxim, & Norris, 2010)</DisplayText><record><rec-number>412</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">412</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Byrnes, Heidi</author><author>Maxim, Hiram. H</author><author>Norris, John M.</author></authors></contributors><titles><title>Realizing advanced FL writing development in collegiate education: Curricular design, pedagogy, assessment</title><secondary-title>Modern Language Journal</secondary-title></titles><periodical><full-title>Modern Language Journal</full-title></periodical><volume>94</volume><number>Monograph Issue</number><dates><year>2010</year></dates><urls><related-urls><url> app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">466</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Byrnes, Heidi</author></authors></contributors><titles><title>The role of task and task-based assessment in a content-oriented collegiate foreign language curriculum</title><secondary-title>Language Testing</secondary-title></titles><periodical><full-title>Language Testing</full-title></periodical><pages>419-437</pages><volume>19</volume><number>4</number><dates><year>2002</year><pub-dates><date>October 1, 2002</date></pub-dates></dates><urls><related-urls><url>;(Byrnes, 2002; Byrnes, Maxim, & Norris, 2010), a timed essay written in class, where students are presented with a topic and have a limited time frame to write their texts, is still the most commonly used assessment tool PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XZWlnbGU8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+NTwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQmFya2FvdWksIDIwMTBhOyBDcnVzYW4sIDIw
MTA7IEN1bW1pbmcsIDIwMTM7IEhhbXAtTHlvbnMsIDIwMTE7IEhlICZhbXA7IFNoaSwgMjAxMjsg
V2VpZ2xlLCAyMDAyKTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj41PC9yZWMtbnVt
YmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYw
cHA5OXJ1c3BheHh4cmZ6ZnN3Ij41PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3
cnRxZ2NBQUFwdk1DRSI+NTM0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJv
b2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XZWlnbGUsIFNh
cmEgQ3VzaGluZzwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs
ZT5Bc3Nlc3Npbmcgd3JpdGluZzwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIwMDI8L3ll
YXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9uPjxwdWJsaXNo
ZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Q3VtbWluZzwvQXV0aG9yPjxZZWFyPjIwMTM8L1ll
YXI+PFJlY051bT40NzA8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ3MDwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBw
OTlydXNwYXh4eHJmemZzdyI+NDcwPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5DdW1taW5nLCBBbGlzdGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPkFzc2Vzc2luZyBpbnRlZ3JhdGVkIHdyaXRpbmcgdGFza3MgZm9yIGFjYWRl
bWljIHB1cnBvc2VzOiBQcm9taXNlcyBhbmQgUGVyaWxzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxl
Pkxhbmd1YWdlIEFzc2Vzc21lbnQgUXVhcnRlcmx5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+
PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TGFuZ3VhZ2UgQXNzZXNzbWVudCBRdWFydGVybHk8L2Z1
bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz4xLTg8L3BhZ2VzPjx2b2x1bWU+MTA8L3ZvbHVt
ZT48bnVtYmVyPjE8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxMzwveWVhcj48cHViLWRhdGVzPjxk
YXRlPjIwMTMvMDEvMDE8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48cHVibGlzaGVyPlJvdXRs
ZWRnZTwvcHVibGlzaGVyPjxpc2JuPjE1NDMtNDMwMzwvaXNibj48dXJscz48cmVsYXRlZC11cmxz
Pjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTA4MC8xNTQzNDMwMy4yMDExLjYyMjAxNjwvdXJs
PjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA4MC8x
NTQzNDMwMy4yMDExLjYyMjAxNjwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PHJlc2VhcmNoLW5v
dGVzPlVzZSB0byBzYXkgd2h5IGludGVncmF0ZWQgYXNzZXNzbWVudCBub3QgYWx3YXlzIHByYWN0
aWNhbC48L3Jlc2VhcmNoLW5vdGVzPjxhY2Nlc3MtZGF0ZT4yMDEzLzA0LzAxPC9hY2Nlc3MtZGF0
ZT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5IZTwvQXV0aG9yPjxZZWFyPjIwMTI8L1ll
YXI+PFJlY051bT40Njg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ2ODwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBw
OTlydXNwYXh4eHJmemZzdyI+NDY4PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5IZSwgTGluZzwvYXV0aG9yPjxhdXRob3I+U2hpLCBMaW5nPC9hdXRob3I+PC9hdXRob3Jz
PjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlRvcGljYWwga25vd2xlZGdlIGFuZCBFU0wg
d3JpdGluZzwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5MYW5ndWFnZSBUZXN0aW5nPC9zZWNvbmRh
cnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TGFuZ3VhZ2UgVGVzdGlu
ZzwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjQ0My00NjQ8L3BhZ2VzPjx2b2x1bWU+
Mjk8L3ZvbHVtZT48bnVtYmVyPjM8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxMjwveWVhcj48cHVi
LWRhdGVzPjxkYXRlPkp1bHkgMSwgMjAxMjwvZGF0ZT48L3B1Yi1kYXRlcz48L2RhdGVzPjx1cmxz
PjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vbHRqLnNhZ2VwdWIuY29tL2NvbnRlbnQvMjkvMy80
NDMuYWJzdHJhY3Q8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3Vy
Y2UtbnVtPjEwLjExNzcvMDI2NTUzMjIxMjQzNjY1OTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+
PHJlc2VhcmNoLW5vdGVzPlVzZSB0byByZWZlcmVuY2UgdGhhdCB0aGUgaW1wcm9tcHR1IGVzc2F5
IHRhc2sgaXMgc3RpbGwgY29tbW9ubHkgdXNlZCA8L3Jlc2VhcmNoLW5vdGVzPjwvcmVjb3JkPjwv
Q2l0ZT48Q2l0ZT48QXV0aG9yPkJhcmthb3VpPC9BdXRob3I+PFllYXI+MjAxMDwvWWVhcj48UmVj
TnVtPjQ2OTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDY5PC9yZWMtbnVtYmVyPjxmb3Jl
aWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3Bh
eHh4cmZ6ZnN3Ij40Njk8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5h
bCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkJh
cmthb3VpLCBLaGFsZWQ8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48
dGl0bGU+RXhwbGFpbmluZyBFU0wgZXNzYXkgaG9saXN0aWMgc2NvcmVzOiBBIG11bHRpbGV2ZWwg
bW9kZWxpbmcgYXBwcm9hY2g8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+TGFuZ3VhZ2UgVGVzdGlu
Zzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkxhbmd1
YWdlIFRlc3Rpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz41MTUtNTM1PC9wYWdl
cz48dm9sdW1lPjI3PC92b2x1bWU+PG51bWJlcj40PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTA8
L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5PY3RvYmVyIDEsIDIwMTA8L2RhdGU+PC9wdWItZGF0ZXM+
PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2x0ai5zYWdlcHViLmNvbS9j
b250ZW50LzI3LzQvNTE1LmFic3RyYWN0PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVj
dHJvbmljLXJlc291cmNlLW51bT4xMC4xMTc3LzAyNjU1MzIyMTAzNjg3MTc8L2VsZWN0cm9uaWMt
cmVzb3VyY2UtbnVtPjxyZXNlYXJjaC1ub3Rlcz5Vc2UgdG8gY29uZmlybSBob3cgaW1wcm9tcHR5
IGVzc2F5IGlzIHN0aWxsIGNvbW1vbi4gQWxzbywgaG93IHNjb3JlIGFuYWx5c2lzIGlzIG9mdGVu
IGRvbmUgd2l0aCB0d28gcmF0aW5nIHNjYWxlcyAoaG9saXN0YyB2ZXJzdXMgYW5hbHl0aWMpLiA8
L3Jlc2VhcmNoLW5vdGVzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkhhbXAtTHlvbnM8
L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFyPjxSZWNOdW0+NDc3PC9SZWNOdW0+PHJlY29yZD48cmVj
LW51bWJlcj40Nzc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlk
PSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQ3Nzwva2V5PjwvZm9yZWln
bi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29u
dHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+SGFtcC1MeW9ucywgTGl6PC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPldyaXRpbmcgYXNzZXNzbWVudDogU2hp
ZnRpbmcgaXNzdWVzLCBuZXcgdG9vbHMsIGVuZHVyaW5nIHF1ZXN0aW9uczwvdGl0bGU+PHNlY29u
ZGFyeS10aXRsZT5Bc3Nlc3NpbmcgV3JpdGluZzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxw
ZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFzc2Vzc2luZyBXcml0aW5nPC9mdWxsLXRpdGxlPjwvcGVy
aW9kaWNhbD48cGFnZXM+My01PC9wYWdlcz48dm9sdW1lPjE2PC92b2x1bWU+PG51bWJlcj4xPC9u
dW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT4xLy88L2RhdGU+
PC9wdWItZGF0ZXM+PC9kYXRlcz48aXNibj4xMDc1LTI5MzU8L2lzYm4+PHVybHM+PHJlbGF0ZWQt
dXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20vc2NpZW5jZS9hcnRpY2xlL3Bp
aS9TMTA3NTI5MzUxMDAwMDU2NTwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25p
Yy1yZXNvdXJjZS1udW0+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAxNi9qLmFzdy4yMDEwLjEyLjAw
MTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PHJlc2VhcmNoLW5vdGVzPlVzZSB0byBiYWNrIHRv
IHN1cHBvcnQgaWRlYSB0aGF0IHdoaWxlICZxdW90O25ldyZxdW90OyBhcHBvYWNoZXMgdG8gYXNz
ZXNzbWVudCBvdXQgdGhlcmUsIHRpbWVkIGVzc2F5IHN0aWxsIGNvbW1vbiBiZWNhdXNlIGNhbm5v
dCBhZ3JlZSB0byBjb3JlIHF1ZXN0aW9ucyAoc2VhcmNoIGZvciAmcXVvdDtjb3JlIGJ1c2luZXNz
JnF1b3Q7KTwvcmVzZWFyY2gtbm90ZXM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Q3J1
c2FuPC9BdXRob3I+PFllYXI+MjAxMDwvWWVhcj48UmVjTnVtPjQ1MzwvUmVjTnVtPjxyZWNvcmQ+
PHJlYy1udW1iZXI+NDUzPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij40NTM8L2tleT48L2Zv
cmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y
cz48YXV0aG9ycz48YXV0aG9yPkNydXNhbiwgRGVib3JhaDwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Bc3Nlc3NtZW50IGluIHRoZSBzZWNvbmQgbGFuZ3Vh
Z2Ugd3JpdGluZyBjbGFzc3Jvb208L3RpdGxlPjwvdGl0bGVzPjxkYXRlcz48eWVhcj4yMDEwPC95
ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5Bbm4gQXJib3IsIE1JPC9wdWItbG9jYXRpb24+PHB1
Ymxpc2hlcj5UaGUgVW5pdmVyc2l0eSBvZiBNaWNoaWdhbiBQcmVzczwvcHVibGlzaGVyPjx1cmxz
PjwvdXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XZWlnbGU8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+NTwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQmFya2FvdWksIDIwMTBhOyBDcnVzYW4sIDIw
MTA7IEN1bW1pbmcsIDIwMTM7IEhhbXAtTHlvbnMsIDIwMTE7IEhlICZhbXA7IFNoaSwgMjAxMjsg
V2VpZ2xlLCAyMDAyKTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj41PC9yZWMtbnVt
YmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYw
cHA5OXJ1c3BheHh4cmZ6ZnN3Ij41PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3
cnRxZ2NBQUFwdk1DRSI+NTM0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJv
b2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XZWlnbGUsIFNh
cmEgQ3VzaGluZzwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs
ZT5Bc3Nlc3Npbmcgd3JpdGluZzwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIwMDI8L3ll
YXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9uPjxwdWJsaXNo
ZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Q3VtbWluZzwvQXV0aG9yPjxZZWFyPjIwMTM8L1ll
YXI+PFJlY051bT40NzA8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ3MDwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBw
OTlydXNwYXh4eHJmemZzdyI+NDcwPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5DdW1taW5nLCBBbGlzdGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPkFzc2Vzc2luZyBpbnRlZ3JhdGVkIHdyaXRpbmcgdGFza3MgZm9yIGFjYWRl
bWljIHB1cnBvc2VzOiBQcm9taXNlcyBhbmQgUGVyaWxzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxl
Pkxhbmd1YWdlIEFzc2Vzc21lbnQgUXVhcnRlcmx5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+
PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TGFuZ3VhZ2UgQXNzZXNzbWVudCBRdWFydGVybHk8L2Z1
bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz4xLTg8L3BhZ2VzPjx2b2x1bWU+MTA8L3ZvbHVt
ZT48bnVtYmVyPjE8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxMzwveWVhcj48cHViLWRhdGVzPjxk
YXRlPjIwMTMvMDEvMDE8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48cHVibGlzaGVyPlJvdXRs
ZWRnZTwvcHVibGlzaGVyPjxpc2JuPjE1NDMtNDMwMzwvaXNibj48dXJscz48cmVsYXRlZC11cmxz
Pjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTA4MC8xNTQzNDMwMy4yMDExLjYyMjAxNjwvdXJs
PjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA4MC8x
NTQzNDMwMy4yMDExLjYyMjAxNjwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PHJlc2VhcmNoLW5v
dGVzPlVzZSB0byBzYXkgd2h5IGludGVncmF0ZWQgYXNzZXNzbWVudCBub3QgYWx3YXlzIHByYWN0
aWNhbC48L3Jlc2VhcmNoLW5vdGVzPjxhY2Nlc3MtZGF0ZT4yMDEzLzA0LzAxPC9hY2Nlc3MtZGF0
ZT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5IZTwvQXV0aG9yPjxZZWFyPjIwMTI8L1ll
YXI+PFJlY051bT40Njg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ2ODwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBw
OTlydXNwYXh4eHJmemZzdyI+NDY4PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5IZSwgTGluZzwvYXV0aG9yPjxhdXRob3I+U2hpLCBMaW5nPC9hdXRob3I+PC9hdXRob3Jz
PjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlRvcGljYWwga25vd2xlZGdlIGFuZCBFU0wg
d3JpdGluZzwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5MYW5ndWFnZSBUZXN0aW5nPC9zZWNvbmRh
cnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TGFuZ3VhZ2UgVGVzdGlu
ZzwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjQ0My00NjQ8L3BhZ2VzPjx2b2x1bWU+
Mjk8L3ZvbHVtZT48bnVtYmVyPjM8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxMjwveWVhcj48cHVi
LWRhdGVzPjxkYXRlPkp1bHkgMSwgMjAxMjwvZGF0ZT48L3B1Yi1kYXRlcz48L2RhdGVzPjx1cmxz
PjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vbHRqLnNhZ2VwdWIuY29tL2NvbnRlbnQvMjkvMy80
NDMuYWJzdHJhY3Q8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3Vy
Y2UtbnVtPjEwLjExNzcvMDI2NTUzMjIxMjQzNjY1OTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+
PHJlc2VhcmNoLW5vdGVzPlVzZSB0byByZWZlcmVuY2UgdGhhdCB0aGUgaW1wcm9tcHR1IGVzc2F5
IHRhc2sgaXMgc3RpbGwgY29tbW9ubHkgdXNlZCA8L3Jlc2VhcmNoLW5vdGVzPjwvcmVjb3JkPjwv
Q2l0ZT48Q2l0ZT48QXV0aG9yPkJhcmthb3VpPC9BdXRob3I+PFllYXI+MjAxMDwvWWVhcj48UmVj
TnVtPjQ2OTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDY5PC9yZWMtbnVtYmVyPjxmb3Jl
aWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3Bh
eHh4cmZ6ZnN3Ij40Njk8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5h
bCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkJh
cmthb3VpLCBLaGFsZWQ8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48
dGl0bGU+RXhwbGFpbmluZyBFU0wgZXNzYXkgaG9saXN0aWMgc2NvcmVzOiBBIG11bHRpbGV2ZWwg
bW9kZWxpbmcgYXBwcm9hY2g8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+TGFuZ3VhZ2UgVGVzdGlu
Zzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkxhbmd1
YWdlIFRlc3Rpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz41MTUtNTM1PC9wYWdl
cz48dm9sdW1lPjI3PC92b2x1bWU+PG51bWJlcj40PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTA8
L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5PY3RvYmVyIDEsIDIwMTA8L2RhdGU+PC9wdWItZGF0ZXM+
PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2x0ai5zYWdlcHViLmNvbS9j
b250ZW50LzI3LzQvNTE1LmFic3RyYWN0PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVj
dHJvbmljLXJlc291cmNlLW51bT4xMC4xMTc3LzAyNjU1MzIyMTAzNjg3MTc8L2VsZWN0cm9uaWMt
cmVzb3VyY2UtbnVtPjxyZXNlYXJjaC1ub3Rlcz5Vc2UgdG8gY29uZmlybSBob3cgaW1wcm9tcHR5
IGVzc2F5IGlzIHN0aWxsIGNvbW1vbi4gQWxzbywgaG93IHNjb3JlIGFuYWx5c2lzIGlzIG9mdGVu
IGRvbmUgd2l0aCB0d28gcmF0aW5nIHNjYWxlcyAoaG9saXN0YyB2ZXJzdXMgYW5hbHl0aWMpLiA8
L3Jlc2VhcmNoLW5vdGVzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkhhbXAtTHlvbnM8
L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFyPjxSZWNOdW0+NDc3PC9SZWNOdW0+PHJlY29yZD48cmVj
LW51bWJlcj40Nzc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlk
PSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQ3Nzwva2V5PjwvZm9yZWln
bi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29u
dHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+SGFtcC1MeW9ucywgTGl6PC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPldyaXRpbmcgYXNzZXNzbWVudDogU2hp
ZnRpbmcgaXNzdWVzLCBuZXcgdG9vbHMsIGVuZHVyaW5nIHF1ZXN0aW9uczwvdGl0bGU+PHNlY29u
ZGFyeS10aXRsZT5Bc3Nlc3NpbmcgV3JpdGluZzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxw
ZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFzc2Vzc2luZyBXcml0aW5nPC9mdWxsLXRpdGxlPjwvcGVy
aW9kaWNhbD48cGFnZXM+My01PC9wYWdlcz48dm9sdW1lPjE2PC92b2x1bWU+PG51bWJlcj4xPC9u
dW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT4xLy88L2RhdGU+
PC9wdWItZGF0ZXM+PC9kYXRlcz48aXNibj4xMDc1LTI5MzU8L2lzYm4+PHVybHM+PHJlbGF0ZWQt
dXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20vc2NpZW5jZS9hcnRpY2xlL3Bp
aS9TMTA3NTI5MzUxMDAwMDU2NTwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25p
Yy1yZXNvdXJjZS1udW0+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAxNi9qLmFzdy4yMDEwLjEyLjAw
MTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PHJlc2VhcmNoLW5vdGVzPlVzZSB0byBiYWNrIHRv
IHN1cHBvcnQgaWRlYSB0aGF0IHdoaWxlICZxdW90O25ldyZxdW90OyBhcHBvYWNoZXMgdG8gYXNz
ZXNzbWVudCBvdXQgdGhlcmUsIHRpbWVkIGVzc2F5IHN0aWxsIGNvbW1vbiBiZWNhdXNlIGNhbm5v
dCBhZ3JlZSB0byBjb3JlIHF1ZXN0aW9ucyAoc2VhcmNoIGZvciAmcXVvdDtjb3JlIGJ1c2luZXNz
JnF1b3Q7KTwvcmVzZWFyY2gtbm90ZXM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Q3J1
c2FuPC9BdXRob3I+PFllYXI+MjAxMDwvWWVhcj48UmVjTnVtPjQ1MzwvUmVjTnVtPjxyZWNvcmQ+
PHJlYy1udW1iZXI+NDUzPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij40NTM8L2tleT48L2Zv
cmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y
cz48YXV0aG9ycz48YXV0aG9yPkNydXNhbiwgRGVib3JhaDwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Bc3Nlc3NtZW50IGluIHRoZSBzZWNvbmQgbGFuZ3Vh
Z2Ugd3JpdGluZyBjbGFzc3Jvb208L3RpdGxlPjwvdGl0bGVzPjxkYXRlcz48eWVhcj4yMDEwPC95
ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5Bbm4gQXJib3IsIE1JPC9wdWItbG9jYXRpb24+PHB1
Ymxpc2hlcj5UaGUgVW5pdmVyc2l0eSBvZiBNaWNoaWdhbiBQcmVzczwvcHVibGlzaGVyPjx1cmxz
PjwvdXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Barkaoui, 2010a; Crusan, 2010; Cumming, 2013; Hamp-Lyons, 2011; He & Shi, 2012; Weigle, 2002). Even writing assessment portfolios usually include at least one piece of timed writing ADDIN EN.CITE <EndNote><Cite><Author>Crusan</Author><Year>2010</Year><RecNum>453</RecNum><DisplayText>(Crusan, 2010; White, 1984)</DisplayText><record><rec-number>453</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">453</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Crusan, Deborah</author></authors></contributors><titles><title>Assessment in the second language writing classroom</title></titles><dates><year>2010</year></dates><pub-location>Ann Arbor, MI</pub-location><publisher>The University of Michigan Press</publisher><urls></urls></record></Cite><Cite><Author>White</Author><Year>1984</Year><RecNum>483</RecNum><record><rec-number>483</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">483</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>White, Edward M.</author></authors></contributors><titles><title>Holisticism</title><secondary-title>College Composition and Communication</secondary-title></titles><periodical><full-title>College Composition and Communication</full-title></periodical><pages>400-409</pages><volume>35</volume><number>4</number><dates><year>1984</year></dates><publisher>National Council of Teachers of English</publisher><isbn>0010096X</isbn><urls><related-urls><url>;(Crusan, 2010; White, 1984). The continued use of the essay exam is due at least in part to two advantages of the timed essay: (a) One knows for certain who the writer is and that students submit their own writing ADDIN EN.CITE <EndNote><Cite><Author>Weigle</Author><Year>2012</Year><RecNum>480</RecNum><DisplayText>(Weigle, 2012; White, 1995)</DisplayText><record><rec-number>480</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">480</key></foreign-keys><ref-type name="Book Section">5</ref-type><contributors><authors><author>Weigle, Sara Cushing</author></authors><secondary-authors><author>Coombe, Christine</author><author>Davidson, Peter</author><author>O'Sullivan, Barry</author><author>Stoynoff, Stephen</author></secondary-authors></contributors><titles><title>Assessing writing</title><secondary-title>The Cambridge guide to second language assessment</secondary-title></titles><pages>218-224</pages><dates><year>2012</year></dates><pub-location>Cambridge</pub-location><publisher>Cambridge University Press</publisher><urls></urls></record></Cite><Cite><Author>White</Author><Year>1995</Year><RecNum>482</RecNum><record><rec-number>482</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">482</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>White, Edward M.</author></authors></contributors><titles><title>An apologia for the timed impromptu essay test</title><secondary-title>College Composition and Communication</secondary-title></titles><periodical><full-title>College Composition and Communication</full-title></periodical><pages>30-45</pages><volume>46</volume><number>1</number><dates><year>1995</year></dates><publisher>National Council of Teachers of English</publisher><isbn>0010096X</isbn><urls><related-urls><url>;(Weigle, 2012; White, 1995), and (b) one obtains more accurate information about students’ language ability ADDIN EN.CITE <EndNote><Cite><Author>Weigle</Author><Year>2012</Year><RecNum>480</RecNum><DisplayText>(Weigle, 2012)</DisplayText><record><rec-number>480</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">480</key></foreign-keys><ref-type name="Book Section">5</ref-type><contributors><authors><author>Weigle, Sara Cushing</author></authors><secondary-authors><author>Coombe, Christine</author><author>Davidson, Peter</author><author>O'Sullivan, Barry</author><author>Stoynoff, Stephen</author></secondary-authors></contributors><titles><title>Assessing writing</title><secondary-title>The Cambridge guide to second language assessment</secondary-title></titles><pages>218-224</pages><dates><year>2012</year></dates><pub-location>Cambridge</pub-location><publisher>Cambridge University Press</publisher><urls></urls></record></Cite></EndNote>(Weigle, 2012). Student performance on an essay task has to be evaluated by qualified judges or teachers themselves ADDIN EN.CITE <EndNote><Cite><Author>Norris</Author><Year>1998</Year><RecNum>121</RecNum><DisplayText>(Norris, Brown, Hudson, & Yoshioka, 1998)</DisplayText><record><rec-number>121</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">121</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">632</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Norris, John M.</author><author>Brown, J. D.</author><author>Hudson, Thom</author><author>Yoshioka, J.</author></authors></contributors><titles><title>Designing second language performance assessments</title></titles><dates><year>1998</year></dates><pub-location>Honolulu, HI</pub-location><publisher>University of Hawai'i, Second Language Teaching and Curriculum Center</publisher><isbn>18</isbn><work-type>Technical Report</work-type><urls></urls></record></Cite></EndNote>(Norris, Brown, Hudson, & Yoshioka, 1998), usually through the use of some kind of rating scale PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5UdXJuZXI8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+MTc2PC9SZWNOdW0+PERpc3BsYXlUZXh0PihDcnVzYW4sIDIwMTA7IEhhbXAtTHlvbnMs
IDE5OTE7IFR1cm5lciAmYW1wOyBVcHNodXIsIDIwMDI7IFdlaWdsZSwgMjAwMiwgMjAxMik8L0Rp
c3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MTc2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl
eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6
ZnN3Ij4xNzY8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNF
Ij42ODM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xl
Ij4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlR1cm5lciwgQ2Fy
b2x5biBFLjwvYXV0aG9yPjxhdXRob3I+VXBzaHVyLCBKb2huIEEuPC9hdXRob3I+PC9hdXRob3Jz
PjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlJhdGluZyBzY2FsZXMgZGVyaXZlZCBmcm9t
IHN0dWRlbnQgc2FtcGxlczogRWZmZWN0cyBvZiB0aGUgc2NhbGUgbWFrZXIgYW5kIHRoZSBzdHVk
ZW50IHNhbXBsZSBvbiBzY2FsZSBjb250ZW50IGFuZCBzdHVkZW50IHNjb3JlczwvdGl0bGU+PHNl
Y29uZGFyeS10aXRsZT5URVNPTCBRdWFydGVybHk8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48
cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5URVNPTCBRdWFydGVybHk8L2Z1bGwtdGl0bGU+PC9wZXJp
b2RpY2FsPjxwYWdlcz40OS03MDwvcGFnZXM+PHZvbHVtZT4zNjwvdm9sdW1lPjxudW1iZXI+MTwv
bnVtYmVyPjxkYXRlcz48eWVhcj4yMDAyPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjxlbGVj
dHJvbmljLXJlc291cmNlLW51bT4xMC4yMzA3LzM1ODgzNjA8L2VsZWN0cm9uaWMtcmVzb3VyY2Ut
bnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkNydXNhbjwvQXV0aG9yPjxZZWFyPjIw
MTA8L1llYXI+PFJlY051bT40NTM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ1MzwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3
ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NDUzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl
IG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5D
cnVzYW4sIERlYm9yYWg8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48
dGl0bGU+QXNzZXNzbWVudCBpbiB0aGUgc2Vjb25kIGxhbmd1YWdlIHdyaXRpbmcgY2xhc3Nyb29t
PC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MjAxMDwveWVhcj48L2RhdGVzPjxwdWItbG9j
YXRpb24+QW5uIEFyYm9yLCBNSTwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+VGhlIFVuaXZlcnNp
dHkgb2YgTWljaGlnYW4gUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9D
aXRlPjxDaXRlPjxBdXRob3I+V2VpZ2xlPC9BdXRob3I+PFllYXI+MjAxMjwvWWVhcj48UmVjTnVt
PjQ4MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDgwPC9yZWMtbnVtYmVyPjxmb3JlaWdu
LWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4
cmZ6ZnN3Ij40ODA8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayBTZWN0
aW9uIj41PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+V2VpZ2xlLCBT
YXJhIEN1c2hpbmc8L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1hdXRob3JzPjxhdXRob3I+
Q29vbWJlLCBDaHJpc3RpbmU8L2F1dGhvcj48YXV0aG9yPkRhdmlkc29uLCBQZXRlcjwvYXV0aG9y
PjxhdXRob3I+TyZhcG9zO1N1bGxpdmFuLCBCYXJyeTwvYXV0aG9yPjxhdXRob3I+U3RveW5vZmYs
IFN0ZXBoZW48L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRs
ZXM+PHRpdGxlPkFzc2Vzc2luZyB3cml0aW5nPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlRoZSBD
YW1icmlkZ2UgZ3VpZGUgdG8gc2Vjb25kIGxhbmd1YWdlIGFzc2Vzc21lbnQ8L3NlY29uZGFyeS10
aXRsZT48L3RpdGxlcz48cGFnZXM+MjE4LTIyNDwvcGFnZXM+PGRhdGVzPjx5ZWFyPjIwMTI8L3ll
YXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9uPjxwdWJsaXNo
ZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+SGFtcC1MeW9uczwvQXV0aG9yPjxZZWFyPjE5OTE8
L1llYXI+PFJlY051bT4yNDQ8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjI0NDwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InZzMDJldnpyaWVzcjI3ZWU5
YWQ1dnhwczJ0dmQyZjAwMjByMiI+MjQ0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5h
bWU9IkJvb2sgU2VjdGlvbiI+NTwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkhhbXAtTHlvbnMsIExpejwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1dGhvcnM+
PGF1dGhvcj5IYW1wLUx5b25zLCBMaXo8L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29u
dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPldoYXQgaXMgYSAmcXVvdDt3cml0aW5nIHRlc3QmcXVv
dDs/PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFzc2Vzc2luZyBzZWNvbmQgbGFuZ3VhZ2Ugd3Jp
dGluZyBpbiBhY2FkZW1pYyBjb250ZXh0czwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdl
cz41LTE1PC9wYWdlcz48ZGF0ZXM+PHllYXI+MTk5MTwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRp
b24+Tm9yd29vZCwgTko8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPkFibGV4PC9wdWJsaXNoZXI+
PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPldlaWdsZTwvQXV0aG9y
PjxZZWFyPjIwMDI8L1llYXI+PFJlY051bT41PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj41
PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2
d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij41PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1p
ZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl
IG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5X
ZWlnbGUsIFNhcmEgQ3VzaGluZzwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0
bGVzPjx0aXRsZT5Bc3Nlc3Npbmcgd3JpdGluZzwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFy
PjIwMDI8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9u
PjxwdWJsaXNoZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48
L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT4A
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5UdXJuZXI8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+MTc2PC9SZWNOdW0+PERpc3BsYXlUZXh0PihDcnVzYW4sIDIwMTA7IEhhbXAtTHlvbnMs
IDE5OTE7IFR1cm5lciAmYW1wOyBVcHNodXIsIDIwMDI7IFdlaWdsZSwgMjAwMiwgMjAxMik8L0Rp
c3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MTc2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl
eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6
ZnN3Ij4xNzY8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNF
Ij42ODM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xl
Ij4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlR1cm5lciwgQ2Fy
b2x5biBFLjwvYXV0aG9yPjxhdXRob3I+VXBzaHVyLCBKb2huIEEuPC9hdXRob3I+PC9hdXRob3Jz
PjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlJhdGluZyBzY2FsZXMgZGVyaXZlZCBmcm9t
IHN0dWRlbnQgc2FtcGxlczogRWZmZWN0cyBvZiB0aGUgc2NhbGUgbWFrZXIgYW5kIHRoZSBzdHVk
ZW50IHNhbXBsZSBvbiBzY2FsZSBjb250ZW50IGFuZCBzdHVkZW50IHNjb3JlczwvdGl0bGU+PHNl
Y29uZGFyeS10aXRsZT5URVNPTCBRdWFydGVybHk8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48
cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5URVNPTCBRdWFydGVybHk8L2Z1bGwtdGl0bGU+PC9wZXJp
b2RpY2FsPjxwYWdlcz40OS03MDwvcGFnZXM+PHZvbHVtZT4zNjwvdm9sdW1lPjxudW1iZXI+MTwv
bnVtYmVyPjxkYXRlcz48eWVhcj4yMDAyPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjxlbGVj
dHJvbmljLXJlc291cmNlLW51bT4xMC4yMzA3LzM1ODgzNjA8L2VsZWN0cm9uaWMtcmVzb3VyY2Ut
bnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkNydXNhbjwvQXV0aG9yPjxZZWFyPjIw
MTA8L1llYXI+PFJlY051bT40NTM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ1MzwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3
ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NDUzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl
IG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5D
cnVzYW4sIERlYm9yYWg8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48
dGl0bGU+QXNzZXNzbWVudCBpbiB0aGUgc2Vjb25kIGxhbmd1YWdlIHdyaXRpbmcgY2xhc3Nyb29t
PC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MjAxMDwveWVhcj48L2RhdGVzPjxwdWItbG9j
YXRpb24+QW5uIEFyYm9yLCBNSTwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+VGhlIFVuaXZlcnNp
dHkgb2YgTWljaGlnYW4gUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9D
aXRlPjxDaXRlPjxBdXRob3I+V2VpZ2xlPC9BdXRob3I+PFllYXI+MjAxMjwvWWVhcj48UmVjTnVt
PjQ4MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDgwPC9yZWMtbnVtYmVyPjxmb3JlaWdu
LWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4
cmZ6ZnN3Ij40ODA8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayBTZWN0
aW9uIj41PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+V2VpZ2xlLCBT
YXJhIEN1c2hpbmc8L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1hdXRob3JzPjxhdXRob3I+
Q29vbWJlLCBDaHJpc3RpbmU8L2F1dGhvcj48YXV0aG9yPkRhdmlkc29uLCBQZXRlcjwvYXV0aG9y
PjxhdXRob3I+TyZhcG9zO1N1bGxpdmFuLCBCYXJyeTwvYXV0aG9yPjxhdXRob3I+U3RveW5vZmYs
IFN0ZXBoZW48L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRs
ZXM+PHRpdGxlPkFzc2Vzc2luZyB3cml0aW5nPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlRoZSBD
YW1icmlkZ2UgZ3VpZGUgdG8gc2Vjb25kIGxhbmd1YWdlIGFzc2Vzc21lbnQ8L3NlY29uZGFyeS10
aXRsZT48L3RpdGxlcz48cGFnZXM+MjE4LTIyNDwvcGFnZXM+PGRhdGVzPjx5ZWFyPjIwMTI8L3ll
YXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9uPjxwdWJsaXNo
ZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+SGFtcC1MeW9uczwvQXV0aG9yPjxZZWFyPjE5OTE8
L1llYXI+PFJlY051bT4yNDQ8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjI0NDwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InZzMDJldnpyaWVzcjI3ZWU5
YWQ1dnhwczJ0dmQyZjAwMjByMiI+MjQ0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5h
bWU9IkJvb2sgU2VjdGlvbiI+NTwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkhhbXAtTHlvbnMsIExpejwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1dGhvcnM+
PGF1dGhvcj5IYW1wLUx5b25zLCBMaXo8L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29u
dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPldoYXQgaXMgYSAmcXVvdDt3cml0aW5nIHRlc3QmcXVv
dDs/PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFzc2Vzc2luZyBzZWNvbmQgbGFuZ3VhZ2Ugd3Jp
dGluZyBpbiBhY2FkZW1pYyBjb250ZXh0czwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdl
cz41LTE1PC9wYWdlcz48ZGF0ZXM+PHllYXI+MTk5MTwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRp
b24+Tm9yd29vZCwgTko8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPkFibGV4PC9wdWJsaXNoZXI+
PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPldlaWdsZTwvQXV0aG9y
PjxZZWFyPjIwMDI8L1llYXI+PFJlY051bT41PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj41
PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2
d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij41PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1p
ZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl
IG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5X
ZWlnbGUsIFNhcmEgQ3VzaGluZzwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0
bGVzPjx0aXRsZT5Bc3Nlc3Npbmcgd3JpdGluZzwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFy
PjIwMDI8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9u
PjxwdWJsaXNoZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48
L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT4A
ADDIN EN.CITE.DATA (Crusan, 2010; Hamp-Lyons, 1991; Turner & Upshur, 2002; Weigle, 2002, 2012). While it may be true that rating scales or rubrics are not used in all contexts, it is also certain that they should always be used to ensure good writing assessment. Rating scales appropriate for the assessment task improve reliability through clear articulation of assessment criteria for the benefit of both teachers and students PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XZWlnbGU8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+NTwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQ3J1c2FuLCAyMDEwOyBXZWlnbGUsIDIwMDIs
IDIwMTI7IFdoaXRlLCAxOTg0KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj41PC9y
ZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2Vm
NXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij41PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0i
U2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5h
bWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XZWln
bGUsIFNhcmEgQ3VzaGluZzwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5Bc3Nlc3Npbmcgd3JpdGluZzwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIw
MDI8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9uPjxw
dWJsaXNoZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3Vy
bHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V2VpZ2xlPC9BdXRob3I+PFllYXI+MjAx
MjwvWWVhcj48UmVjTnVtPjQ4MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDgwPC9yZWMt
bnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdl
YXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij40ODA8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUg
bmFtZT0iQm9vayBTZWN0aW9uIj41PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+V2VpZ2xlLCBTYXJhIEN1c2hpbmc8L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1h
dXRob3JzPjxhdXRob3I+Q29vbWJlLCBDaHJpc3RpbmU8L2F1dGhvcj48YXV0aG9yPkRhdmlkc29u
LCBQZXRlcjwvYXV0aG9yPjxhdXRob3I+TyZhcG9zO1N1bGxpdmFuLCBCYXJyeTwvYXV0aG9yPjxh
dXRob3I+U3RveW5vZmYsIFN0ZXBoZW48L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29u
dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc2Vzc2luZyB3cml0aW5nPC90aXRsZT48c2Vjb25k
YXJ5LXRpdGxlPlRoZSBDYW1icmlkZ2UgZ3VpZGUgdG8gc2Vjb25kIGxhbmd1YWdlIGFzc2Vzc21l
bnQ8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MjE4LTIyNDwvcGFnZXM+PGRhdGVz
Pjx5ZWFyPjIwMTI8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxv
Y2F0aW9uPjxwdWJsaXNoZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48
dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V2hpdGU8L0F1dGhvcj48
WWVhcj4xOTg0PC9ZZWFyPjxSZWNOdW0+NDgzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40
ODM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5
NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQ4Mzwva2V5PjwvZm9yZWlnbi1rZXlzPjxy
ZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3Jz
PjxhdXRob3JzPjxhdXRob3I+V2hpdGUsIEVkd2FyZCBNLjwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Ib2xpc3RpY2lzbTwvdGl0bGU+PHNlY29uZGFyeS10
aXRsZT5Db2xsZWdlIENvbXBvc2l0aW9uIGFuZCBDb21tdW5pY2F0aW9uPC9zZWNvbmRhcnktdGl0
bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+Q29sbGVnZSBDb21wb3NpdGlvbiBh
bmQgQ29tbXVuaWNhdGlvbjwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjQwMC00MDk8
L3BhZ2VzPjx2b2x1bWU+MzU8L3ZvbHVtZT48bnVtYmVyPjQ8L251bWJlcj48ZGF0ZXM+PHllYXI+
MTk4NDwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+TmF0aW9uYWwgQ291bmNpbCBvZiBUZWFjaGVy
cyBvZiBFbmdsaXNoPC9wdWJsaXNoZXI+PGlzYm4+MDAxMDA5Nlg8L2lzYm4+PHVybHM+PHJlbGF0
ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuanN0b3Iub3JnL3N0YWJsZS8zNTc3OTI8L3VybD48L3Jl
bGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjIzMDcvMzU3Nzky
PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5D
cnVzYW48L0F1dGhvcj48WWVhcj4yMDEwPC9ZZWFyPjxSZWNOdW0+NDUzPC9SZWNOdW0+PHJlY29y
ZD48cmVjLW51bWJlcj40NTM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4i
IGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQ1Mzwva2V5Pjwv
Zm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0
b3JzPjxhdXRob3JzPjxhdXRob3I+Q3J1c2FuLCBEZWJvcmFoPC9hdXRob3I+PC9hdXRob3JzPjwv
Y29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc2Vzc21lbnQgaW4gdGhlIHNlY29uZCBsYW5n
dWFnZSB3cml0aW5nIGNsYXNzcm9vbTwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIwMTA8
L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkFubiBBcmJvciwgTUk8L3B1Yi1sb2NhdGlvbj48
cHVibGlzaGVyPlRoZSBVbml2ZXJzaXR5IG9mIE1pY2hpZ2FuIFByZXNzPC9wdWJsaXNoZXI+PHVy
bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XZWlnbGU8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+NTwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQ3J1c2FuLCAyMDEwOyBXZWlnbGUsIDIwMDIs
IDIwMTI7IFdoaXRlLCAxOTg0KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj41PC9y
ZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2Vm
NXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij41PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0i
U2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5h
bWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XZWln
bGUsIFNhcmEgQ3VzaGluZzwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5Bc3Nlc3Npbmcgd3JpdGluZzwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIw
MDI8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9uPjxw
dWJsaXNoZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3Vy
bHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V2VpZ2xlPC9BdXRob3I+PFllYXI+MjAx
MjwvWWVhcj48UmVjTnVtPjQ4MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDgwPC9yZWMt
bnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdl
YXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij40ODA8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUg
bmFtZT0iQm9vayBTZWN0aW9uIj41PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+V2VpZ2xlLCBTYXJhIEN1c2hpbmc8L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1h
dXRob3JzPjxhdXRob3I+Q29vbWJlLCBDaHJpc3RpbmU8L2F1dGhvcj48YXV0aG9yPkRhdmlkc29u
LCBQZXRlcjwvYXV0aG9yPjxhdXRob3I+TyZhcG9zO1N1bGxpdmFuLCBCYXJyeTwvYXV0aG9yPjxh
dXRob3I+U3RveW5vZmYsIFN0ZXBoZW48L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29u
dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc2Vzc2luZyB3cml0aW5nPC90aXRsZT48c2Vjb25k
YXJ5LXRpdGxlPlRoZSBDYW1icmlkZ2UgZ3VpZGUgdG8gc2Vjb25kIGxhbmd1YWdlIGFzc2Vzc21l
bnQ8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MjE4LTIyNDwvcGFnZXM+PGRhdGVz
Pjx5ZWFyPjIwMTI8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxv
Y2F0aW9uPjxwdWJsaXNoZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48
dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V2hpdGU8L0F1dGhvcj48
WWVhcj4xOTg0PC9ZZWFyPjxSZWNOdW0+NDgzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40
ODM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5
NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQ4Mzwva2V5PjwvZm9yZWlnbi1rZXlzPjxy
ZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3Jz
PjxhdXRob3JzPjxhdXRob3I+V2hpdGUsIEVkd2FyZCBNLjwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Ib2xpc3RpY2lzbTwvdGl0bGU+PHNlY29uZGFyeS10
aXRsZT5Db2xsZWdlIENvbXBvc2l0aW9uIGFuZCBDb21tdW5pY2F0aW9uPC9zZWNvbmRhcnktdGl0
bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+Q29sbGVnZSBDb21wb3NpdGlvbiBh
bmQgQ29tbXVuaWNhdGlvbjwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjQwMC00MDk8
L3BhZ2VzPjx2b2x1bWU+MzU8L3ZvbHVtZT48bnVtYmVyPjQ8L251bWJlcj48ZGF0ZXM+PHllYXI+
MTk4NDwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+TmF0aW9uYWwgQ291bmNpbCBvZiBUZWFjaGVy
cyBvZiBFbmdsaXNoPC9wdWJsaXNoZXI+PGlzYm4+MDAxMDA5Nlg8L2lzYm4+PHVybHM+PHJlbGF0
ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuanN0b3Iub3JnL3N0YWJsZS8zNTc3OTI8L3VybD48L3Jl
bGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjIzMDcvMzU3Nzky
PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5D
cnVzYW48L0F1dGhvcj48WWVhcj4yMDEwPC9ZZWFyPjxSZWNOdW0+NDUzPC9SZWNOdW0+PHJlY29y
ZD48cmVjLW51bWJlcj40NTM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4i
IGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQ1Mzwva2V5Pjwv
Zm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0
b3JzPjxhdXRob3JzPjxhdXRob3I+Q3J1c2FuLCBEZWJvcmFoPC9hdXRob3I+PC9hdXRob3JzPjwv
Y29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc2Vzc21lbnQgaW4gdGhlIHNlY29uZCBsYW5n
dWFnZSB3cml0aW5nIGNsYXNzcm9vbTwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIwMTA8
L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkFubiBBcmJvciwgTUk8L3B1Yi1sb2NhdGlvbj48
cHVibGlzaGVyPlRoZSBVbml2ZXJzaXR5IG9mIE1pY2hpZ2FuIFByZXNzPC9wdWJsaXNoZXI+PHVy
bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE.DATA (Crusan, 2010; Weigle, 2002, 2012; White, 1984). Rating scales generally belong to one of two categories: holistic scales, where one overall score is assigned for a piece of writing, or analytic scales, where subscores on different criteria make up the overall score. When L1 or L2 writing is rated by means of a holistic rating scale, grammatical accuracy is often correlated to the overall performance assessment PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CYXJrYW91aTwvQXV0aG9yPjxZZWFyPjIwMTA8L1llYXI+
PFJlY051bT4zNjg8L1JlY051bT48RGlzcGxheVRleHQ+KEJhcmthb3VpLCAyMDEwYjsgSHVhbmcg
JmFtcDsgRm9vdGUsIDIwMTA7IFN3ZWVkbGVyLUJyb3duLCAxOTkzKTwvRGlzcGxheVRleHQ+PHJl
Y29yZD48cmVjLW51bWJlcj4zNjg8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i
RU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjM2ODwva2V5
PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYt
dHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+QmFya2FvdWksIEtoYWxlZDwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5WYXJpYWJpbGl0eSBp
biBFU0wgZXNzYXkgcmF0aW5nIHByb2Nlc3NlczogVGhlIHJvbGUgb2YgdGhlIHJhdGluZyBzY2Fs
ZSBhbmQgcmF0ZXIgZXhwZXJpZW5jZTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5MYW5ndWFnZSBB
c3Nlc3NtZW50IFF1YXJ0ZXJseTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2Fs
PjxmdWxsLXRpdGxlPkxhbmd1YWdlIEFzc2Vzc21lbnQgUXVhcnRlcmx5PC9mdWxsLXRpdGxlPjwv
cGVyaW9kaWNhbD48cGFnZXM+NTQgLSA3NDwvcGFnZXM+PHZvbHVtZT43PC92b2x1bWU+PG51bWJl
cj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTA8L3llYXI+PC9kYXRlcz48cHVibGlzaGVyPlJv
dXRsZWRnZTwvcHVibGlzaGVyPjxpc2JuPjE1NDMtNDMwMzwvaXNibj48dXJscz48cmVsYXRlZC11
cmxzPjx1cmw+aHR0cDovL3d3dy5pbmZvcm1hd29ybGQuY29tLzEwLjEwODAvMTU0MzQzMDA5MDM0
NjQ0MTg8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVt
PjEwLjEwODAvMTU0MzQzMDA5MDM0NjQ0MTggPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48YWNj
ZXNzLWRhdGU+TWFyY2ggMjksIDIwMTA8L2FjY2Vzcy1kYXRlPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0
ZT48QXV0aG9yPkh1YW5nPC9BdXRob3I+PFllYXI+MjAxMDwvWWVhcj48UmVjTnVtPjQ0MzwvUmVj
TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQzPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl
eSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij40
NDM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x
NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkh1YW5nLCBKaW55YW48
L2F1dGhvcj48YXV0aG9yPkZvb3RlLCBDaGFuZHJhIEouPC9hdXRob3I+PC9hdXRob3JzPjwvY29u
dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkdyYWRpbmcgYmV0d2VlbiB0aGUgbGluZXM6IFdoYXQg
cmVhbGx5IGltcGFjdHMgcHJvZmVzc29yc+KAmSBob2xpc3RpYyBldmFsdWF0aW9uIG9mIEVTTCBn
cmFkdWF0ZSBzdHVkZW50IHdyaXRpbmc/PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkxhbmd1YWdl
IEFzc2Vzc21lbnQgUXVhcnRlcmx5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGlj
YWw+PGZ1bGwtdGl0bGU+TGFuZ3VhZ2UgQXNzZXNzbWVudCBRdWFydGVybHk8L2Z1bGwtdGl0bGU+
PC9wZXJpb2RpY2FsPjxwYWdlcz4yMTktMjMzPC9wYWdlcz48dm9sdW1lPjc8L3ZvbHVtZT48bnVt
YmVyPjM8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxMDwveWVhcj48cHViLWRhdGVzPjxkYXRlPjIw
MTAvMDcvMzA8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48cHVibGlzaGVyPlJvdXRsZWRnZTwv
cHVibGlzaGVyPjxpc2JuPjE1NDMtNDMwMzwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+
aHR0cDovL3d3dy50YW5kZm9ubGluZS5jb20vZG9pL2Ficy8xMC4xMDgwLzE1NDM0MzAwOTAzNTQw
ODk0PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT4x
MC4xMDgwLzE1NDM0MzAwOTAzNTQwODk0PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48YWNjZXNz
LWRhdGU+MjAxMi8wNS8xNDwvYWNjZXNzLWRhdGU+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+U3dlZWRsZXItQnJvd248L0F1dGhvcj48WWVhcj4xOTkzPC9ZZWFyPjxSZWNOdW0+NDkyPC9S
ZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40OTI8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48
a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ci
PjQ5Mjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUi
PjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+U3dlZWRsZXItQnJv
d24sIENhcm9sIE8uPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp
dGxlPkVTTCBlc3NheSBldmFsdWF0aW9uOiBUaGUgaW5mbHVlbmNlIG9mIHNlbnRlbmNlLWxldmVs
IGFuZCByaGV0b3JpY2FsIGZlYXR1cmVzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwg
b2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVy
aW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNlY29uZCBMYW5ndWFnZSBXcml0aW5nPC9m
dWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+My0xNzwvcGFnZXM+PHZvbHVtZT4yPC92b2x1
bWU+PG51bWJlcj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjE5OTM8L3llYXI+PHB1Yi1kYXRlcz48
ZGF0ZT4xLy88L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48aXNibj4xMDYwLTM3NDM8L2lzYm4+
PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20vc2Np
ZW5jZS9hcnRpY2xlL3BpaS8xMDYwMzc0MzkzOTAwMDNMPC91cmw+PC9yZWxhdGVkLXVybHM+PC91
cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT5odHRwOi8vZHguZG9pLm9yZy8xMC4xMDE2LzEw
NjAtMzc0Myg5Myk5MDAwMy1MPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0Np
dGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CYXJrYW91aTwvQXV0aG9yPjxZZWFyPjIwMTA8L1llYXI+
PFJlY051bT4zNjg8L1JlY051bT48RGlzcGxheVRleHQ+KEJhcmthb3VpLCAyMDEwYjsgSHVhbmcg
JmFtcDsgRm9vdGUsIDIwMTA7IFN3ZWVkbGVyLUJyb3duLCAxOTkzKTwvRGlzcGxheVRleHQ+PHJl
Y29yZD48cmVjLW51bWJlcj4zNjg8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i
RU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjM2ODwva2V5
PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYt
dHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+QmFya2FvdWksIEtoYWxlZDwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5WYXJpYWJpbGl0eSBp
biBFU0wgZXNzYXkgcmF0aW5nIHByb2Nlc3NlczogVGhlIHJvbGUgb2YgdGhlIHJhdGluZyBzY2Fs
ZSBhbmQgcmF0ZXIgZXhwZXJpZW5jZTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5MYW5ndWFnZSBB
c3Nlc3NtZW50IFF1YXJ0ZXJseTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2Fs
PjxmdWxsLXRpdGxlPkxhbmd1YWdlIEFzc2Vzc21lbnQgUXVhcnRlcmx5PC9mdWxsLXRpdGxlPjwv
cGVyaW9kaWNhbD48cGFnZXM+NTQgLSA3NDwvcGFnZXM+PHZvbHVtZT43PC92b2x1bWU+PG51bWJl
cj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTA8L3llYXI+PC9kYXRlcz48cHVibGlzaGVyPlJv
dXRsZWRnZTwvcHVibGlzaGVyPjxpc2JuPjE1NDMtNDMwMzwvaXNibj48dXJscz48cmVsYXRlZC11
cmxzPjx1cmw+aHR0cDovL3d3dy5pbmZvcm1hd29ybGQuY29tLzEwLjEwODAvMTU0MzQzMDA5MDM0
NjQ0MTg8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVt
PjEwLjEwODAvMTU0MzQzMDA5MDM0NjQ0MTggPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48YWNj
ZXNzLWRhdGU+TWFyY2ggMjksIDIwMTA8L2FjY2Vzcy1kYXRlPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0
ZT48QXV0aG9yPkh1YW5nPC9BdXRob3I+PFllYXI+MjAxMDwvWWVhcj48UmVjTnVtPjQ0MzwvUmVj
TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQzPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl
eSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij40
NDM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x
NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkh1YW5nLCBKaW55YW48
L2F1dGhvcj48YXV0aG9yPkZvb3RlLCBDaGFuZHJhIEouPC9hdXRob3I+PC9hdXRob3JzPjwvY29u
dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkdyYWRpbmcgYmV0d2VlbiB0aGUgbGluZXM6IFdoYXQg
cmVhbGx5IGltcGFjdHMgcHJvZmVzc29yc+KAmSBob2xpc3RpYyBldmFsdWF0aW9uIG9mIEVTTCBn
cmFkdWF0ZSBzdHVkZW50IHdyaXRpbmc/PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkxhbmd1YWdl
IEFzc2Vzc21lbnQgUXVhcnRlcmx5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGlj
YWw+PGZ1bGwtdGl0bGU+TGFuZ3VhZ2UgQXNzZXNzbWVudCBRdWFydGVybHk8L2Z1bGwtdGl0bGU+
PC9wZXJpb2RpY2FsPjxwYWdlcz4yMTktMjMzPC9wYWdlcz48dm9sdW1lPjc8L3ZvbHVtZT48bnVt
YmVyPjM8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxMDwveWVhcj48cHViLWRhdGVzPjxkYXRlPjIw
MTAvMDcvMzA8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48cHVibGlzaGVyPlJvdXRsZWRnZTwv
cHVibGlzaGVyPjxpc2JuPjE1NDMtNDMwMzwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+
aHR0cDovL3d3dy50YW5kZm9ubGluZS5jb20vZG9pL2Ficy8xMC4xMDgwLzE1NDM0MzAwOTAzNTQw
ODk0PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT4x
MC4xMDgwLzE1NDM0MzAwOTAzNTQwODk0PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48YWNjZXNz
LWRhdGU+MjAxMi8wNS8xNDwvYWNjZXNzLWRhdGU+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+U3dlZWRsZXItQnJvd248L0F1dGhvcj48WWVhcj4xOTkzPC9ZZWFyPjxSZWNOdW0+NDkyPC9S
ZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40OTI8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48
a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ci
PjQ5Mjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUi
PjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+U3dlZWRsZXItQnJv
d24sIENhcm9sIE8uPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp
dGxlPkVTTCBlc3NheSBldmFsdWF0aW9uOiBUaGUgaW5mbHVlbmNlIG9mIHNlbnRlbmNlLWxldmVs
IGFuZCByaGV0b3JpY2FsIGZlYXR1cmVzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwg
b2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVy
aW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNlY29uZCBMYW5ndWFnZSBXcml0aW5nPC9m
dWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+My0xNzwvcGFnZXM+PHZvbHVtZT4yPC92b2x1
bWU+PG51bWJlcj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjE5OTM8L3llYXI+PHB1Yi1kYXRlcz48
ZGF0ZT4xLy88L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48aXNibj4xMDYwLTM3NDM8L2lzYm4+
PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20vc2Np
ZW5jZS9hcnRpY2xlL3BpaS8xMDYwMzc0MzkzOTAwMDNMPC91cmw+PC9yZWxhdGVkLXVybHM+PC91
cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT5odHRwOi8vZHguZG9pLm9yZy8xMC4xMDE2LzEw
NjAtMzc0Myg5Myk5MDAwMy1MPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0Np
dGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Barkaoui, 2010b; Huang & Foote, 2010; Sweedler-Brown, 1993). When an analytical rating scale is used, grammar is often a separate criterion, but there is little research on what construct teachers base their assessment of grammar for classroom L2 writing assessments. In order to examine the teachers’ construct, it is necessary to look to the testing literature for construct definitions as a point of reference.Attempts to define this construct in the assessment literature within the context of L2 writing assessment emerge from models of writing ability based on cognitive science research for L1 writing PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5IYXllczwvQXV0aG9yPjxZZWFyPjE5ODA8L1llYXI+PFJl
Y051bT43ODwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oSGF5ZXMsIDE5OTY7IEhheWVzICZhbXA7IEZs
b3dlciwgMTk4MDsgU2NhcmRhbWFsaWEgJmFtcDsgQmVyZWl0ZXIsIDE5ODcpPC9EaXNwbGF5VGV4
dD48cmVjb3JkPjxyZWMtbnVtYmVyPjc4PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBh
cHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij43ODwv
a2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjU5Mjwva2V5
PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIFNlY3Rpb24iPjU8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5IYXllcywgSm9obiBSLjwvYXV0aG9yPjxh
dXRob3I+Rmxvd2VyLCBMaW5kYSBTLjwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1dGhv
cnM+PGF1dGhvcj5HcmVnZywgTC4gVy48L2F1dGhvcj48YXV0aG9yPlN0ZWluYmVyZywgRS4gUi48
L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxl
PklkZW50aWZ5aW5nIHRoZSBvcmdhbml6YXRpb24gb2Ygd3JpdGluZyBwcm9jZXNzZXM8L3RpdGxl
PjxzZWNvbmRhcnktdGl0bGU+Q29nbml0aXZlIHByb2Nlc3NlcyBpbiB3cml0aW5nPC9zZWNvbmRh
cnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjMtMzA8L3BhZ2VzPjxrZXl3b3Jkcz48a2V5d29yZD5M
MSB3cml0aW5nIHByb2Nlc3M8L2tleXdvcmQ+PGtleXdvcmQ+bW9kZWw8L2tleXdvcmQ+PC9rZXl3
b3Jkcz48ZGF0ZXM+PHllYXI+MTk4MDwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+SGlsbHNk
YWxlLCBOSjwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+RXJsYmF1bTwvcHVibGlzaGVyPjx1cmxz
PjwvdXJscz48bGFuZ3VhZ2U+RW5nbGlzaDwvbGFuZ3VhZ2U+PC9yZWNvcmQ+PC9DaXRlPjxDaXRl
PjxBdXRob3I+U2NhcmRhbWFsaWE8L0F1dGhvcj48WWVhcj4xOTg3PC9ZZWFyPjxSZWNOdW0+ODA8
L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjgwPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+
PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3
Ij44MDwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjU5
NDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIFNlY3Rpb24iPjU8L3Jl
Zi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5TY2FyZGFtYWxpYSwgTS48L2F1
dGhvcj48YXV0aG9yPkJlcmVpdGVyLCBDLjwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1
dGhvcnM+PGF1dGhvcj5Sb3NlbmJlcmcsIFM8L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwv
Y29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPktub3dsZWRnZSB0ZWxsaW5nIGFuZCBrbm93bGVk
Z2UgdHJhbnNmb3JtaW5nIGluIHdyaXR0ZW4gY29tcG9zaXRpb248L3RpdGxlPjxzZWNvbmRhcnkt
dGl0bGU+QWR2YW5jZXMgaW4gYXBwbGllZCBwc3ljaG9saW5ndWlzdGljcywgVm9sdW1lIDI6IFJl
YWRpbmcsIHdyaXRpbmcsIGFuZCBsYW5ndWFnZSBsZWFybmluZzwvc2Vjb25kYXJ5LXRpdGxlPjwv
dGl0bGVzPjxwYWdlcz4xNDItMTc1PC9wYWdlcz48ZGF0ZXM+PHllYXI+MTk4NzwveWVhcj48L2Rh
dGVzPjxwdWItbG9jYXRpb24+Q2FtYnJpZGdlPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5DYW1i
cmlkZ2UgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjx1cmxzPjwvdXJscz48L3JlY29yZD48
L0NpdGU+PENpdGU+PEF1dGhvcj5IYXllczwvQXV0aG9yPjxZZWFyPjE5OTY8L1llYXI+PFJlY051
bT43MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NzI8L3JlYy1udW1iZXI+PGZvcmVpZ24t
a2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhy
Znpmc3ciPjcyPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1D
RSI+NTg3PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2sgU2VjdGlvbiI+
NTwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkpvaG4gUi4gSGF5ZXM8
L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1hdXRob3JzPjxhdXRob3I+TGV2eSwgQy4gTWlj
aGFlbDwvYXV0aG9yPjxhdXRob3I+UmFuZHNkZWxsLCBTYXJhaDwvYXV0aG9yPjwvc2Vjb25kYXJ5
LWF1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+QSBuZXcgZnJhbWV3b3JrIGZv
ciB1bmRlcnN0YW5kaW5nIGNvZ25pdGlvbiBhbmQgYWZmZWN0IGluIHdyaXRpbmc8L3RpdGxlPjxz
ZWNvbmRhcnktdGl0bGU+VGhlIHNjaWVuY2Ugb2Ygd3JpdGluZzogVGhlb3JpZXMsIG1ldGhvZHMs
IGluZGl2aWR1YWwgZGlmZmVyZW5jZXMsIGFuZCBhcHBsaWNhdGlvbnM8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+MS0yNzwvcGFnZXM+PG51bS12b2xzPjE8L251bS12b2xzPjxrZXl3
b3Jkcz48a2V5d29yZD5SZXZpc2VkIEhheWVzICZhbXA7IEZsb3dlciBtb2RlbDwva2V5d29yZD48
a2V5d29yZD5BZmZlY3QgJmFtcDsgbW90aXZhdGlvbiBpbiB3cml0aW5nPC9rZXl3b3JkPjxrZXl3
b3JkPm1vZGVsIG9mIHJldmlzaW9uIHByb2Nlc3M8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+
PHllYXI+MTk5NjwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+TWFod2FoLCBOSjwvcHViLWxv
Y2F0aW9uPjxwdWJsaXNoZXI+TGF3cmVuY2UgRXJsYmF1bTwvcHVibGlzaGVyPjx1cmxzPjwvdXJs
cz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+RUQgNDQxIDI0OTwvZWxlY3Ryb25pYy1yZXNvdXJj
ZS1udW0+PHJlbW90ZS1kYXRhYmFzZS1uYW1lPkVSSUM8L3JlbW90ZS1kYXRhYmFzZS1uYW1lPjxs
YW5ndWFnZT5FbmdsaXNoPC9sYW5ndWFnZT48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5IYXllczwvQXV0aG9yPjxZZWFyPjE5ODA8L1llYXI+PFJl
Y051bT43ODwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oSGF5ZXMsIDE5OTY7IEhheWVzICZhbXA7IEZs
b3dlciwgMTk4MDsgU2NhcmRhbWFsaWEgJmFtcDsgQmVyZWl0ZXIsIDE5ODcpPC9EaXNwbGF5VGV4
dD48cmVjb3JkPjxyZWMtbnVtYmVyPjc4PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBh
cHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij43ODwv
a2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjU5Mjwva2V5
PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIFNlY3Rpb24iPjU8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5IYXllcywgSm9obiBSLjwvYXV0aG9yPjxh
dXRob3I+Rmxvd2VyLCBMaW5kYSBTLjwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1dGhv
cnM+PGF1dGhvcj5HcmVnZywgTC4gVy48L2F1dGhvcj48YXV0aG9yPlN0ZWluYmVyZywgRS4gUi48
L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxl
PklkZW50aWZ5aW5nIHRoZSBvcmdhbml6YXRpb24gb2Ygd3JpdGluZyBwcm9jZXNzZXM8L3RpdGxl
PjxzZWNvbmRhcnktdGl0bGU+Q29nbml0aXZlIHByb2Nlc3NlcyBpbiB3cml0aW5nPC9zZWNvbmRh
cnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjMtMzA8L3BhZ2VzPjxrZXl3b3Jkcz48a2V5d29yZD5M
MSB3cml0aW5nIHByb2Nlc3M8L2tleXdvcmQ+PGtleXdvcmQ+bW9kZWw8L2tleXdvcmQ+PC9rZXl3
b3Jkcz48ZGF0ZXM+PHllYXI+MTk4MDwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+SGlsbHNk
YWxlLCBOSjwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+RXJsYmF1bTwvcHVibGlzaGVyPjx1cmxz
PjwvdXJscz48bGFuZ3VhZ2U+RW5nbGlzaDwvbGFuZ3VhZ2U+PC9yZWNvcmQ+PC9DaXRlPjxDaXRl
PjxBdXRob3I+U2NhcmRhbWFsaWE8L0F1dGhvcj48WWVhcj4xOTg3PC9ZZWFyPjxSZWNOdW0+ODA8
L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjgwPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+
PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3
Ij44MDwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjU5
NDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIFNlY3Rpb24iPjU8L3Jl
Zi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5TY2FyZGFtYWxpYSwgTS48L2F1
dGhvcj48YXV0aG9yPkJlcmVpdGVyLCBDLjwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1
dGhvcnM+PGF1dGhvcj5Sb3NlbmJlcmcsIFM8L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwv
Y29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPktub3dsZWRnZSB0ZWxsaW5nIGFuZCBrbm93bGVk
Z2UgdHJhbnNmb3JtaW5nIGluIHdyaXR0ZW4gY29tcG9zaXRpb248L3RpdGxlPjxzZWNvbmRhcnkt
dGl0bGU+QWR2YW5jZXMgaW4gYXBwbGllZCBwc3ljaG9saW5ndWlzdGljcywgVm9sdW1lIDI6IFJl
YWRpbmcsIHdyaXRpbmcsIGFuZCBsYW5ndWFnZSBsZWFybmluZzwvc2Vjb25kYXJ5LXRpdGxlPjwv
dGl0bGVzPjxwYWdlcz4xNDItMTc1PC9wYWdlcz48ZGF0ZXM+PHllYXI+MTk4NzwveWVhcj48L2Rh
dGVzPjxwdWItbG9jYXRpb24+Q2FtYnJpZGdlPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5DYW1i
cmlkZ2UgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjx1cmxzPjwvdXJscz48L3JlY29yZD48
L0NpdGU+PENpdGU+PEF1dGhvcj5IYXllczwvQXV0aG9yPjxZZWFyPjE5OTY8L1llYXI+PFJlY051
bT43MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NzI8L3JlYy1udW1iZXI+PGZvcmVpZ24t
a2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhy
Znpmc3ciPjcyPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1D
RSI+NTg3PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2sgU2VjdGlvbiI+
NTwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkpvaG4gUi4gSGF5ZXM8
L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1hdXRob3JzPjxhdXRob3I+TGV2eSwgQy4gTWlj
aGFlbDwvYXV0aG9yPjxhdXRob3I+UmFuZHNkZWxsLCBTYXJhaDwvYXV0aG9yPjwvc2Vjb25kYXJ5
LWF1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+QSBuZXcgZnJhbWV3b3JrIGZv
ciB1bmRlcnN0YW5kaW5nIGNvZ25pdGlvbiBhbmQgYWZmZWN0IGluIHdyaXRpbmc8L3RpdGxlPjxz
ZWNvbmRhcnktdGl0bGU+VGhlIHNjaWVuY2Ugb2Ygd3JpdGluZzogVGhlb3JpZXMsIG1ldGhvZHMs
IGluZGl2aWR1YWwgZGlmZmVyZW5jZXMsIGFuZCBhcHBsaWNhdGlvbnM8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+MS0yNzwvcGFnZXM+PG51bS12b2xzPjE8L251bS12b2xzPjxrZXl3
b3Jkcz48a2V5d29yZD5SZXZpc2VkIEhheWVzICZhbXA7IEZsb3dlciBtb2RlbDwva2V5d29yZD48
a2V5d29yZD5BZmZlY3QgJmFtcDsgbW90aXZhdGlvbiBpbiB3cml0aW5nPC9rZXl3b3JkPjxrZXl3
b3JkPm1vZGVsIG9mIHJldmlzaW9uIHByb2Nlc3M8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+
PHllYXI+MTk5NjwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+TWFod2FoLCBOSjwvcHViLWxv
Y2F0aW9uPjxwdWJsaXNoZXI+TGF3cmVuY2UgRXJsYmF1bTwvcHVibGlzaGVyPjx1cmxzPjwvdXJs
cz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+RUQgNDQxIDI0OTwvZWxlY3Ryb25pYy1yZXNvdXJj
ZS1udW0+PHJlbW90ZS1kYXRhYmFzZS1uYW1lPkVSSUM8L3JlbW90ZS1kYXRhYmFzZS1uYW1lPjxs
YW5ndWFnZT5FbmdsaXNoPC9sYW5ndWFnZT48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE.DATA (Hayes, 1996; Hayes & Flower, 1980; Scardamalia & Bereiter, 1987) or the writing and testing literature for L2 writing PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XZWlnbGU8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+NTwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQmFjaG1hbiwgMTk5MDsgQmFjaG1hbiAmYW1w
OyBQYWxtZXIsIDE5OTY7IEdyYWJlICZhbXA7IEthcGxhbiwgMTk5NjsgV2VpZ2xlLCAyMDAyKTwv
RGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj41PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl
eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6
ZnN3Ij41PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+
NTM0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XZWlnbGUsIFNhcmEgQ3VzaGluZzwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Bc3Nlc3Npbmcgd3Jp
dGluZzwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIwMDI8L3llYXI+PC9kYXRlcz48cHVi
LWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+Q2FtYnJpZGdlIFVu
aXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD
aXRlPjxBdXRob3I+QmFjaG1hbjwvQXV0aG9yPjxZZWFyPjE5OTA8L1llYXI+PFJlY051bT4yPC9S
ZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl
eSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4y
PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTMxPC9r
ZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5CYWNobWFuLCBMeWxlIEYuPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkZ1bmRhbWVudGFsIGNvbnNpZGVyYXRp
b25zIGluIGxhbmd1YWdlIHRlc3RpbmcuPC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MTk5
MDwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+T3hmb3JkPC9wdWItbG9jYXRpb24+PHB1Ymxp
c2hlcj5PeGZvcmQgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjx1cmxzPjwvdXJscz48L3Jl
Y29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5CYWNobWFuPC9BdXRob3I+PFllYXI+MTk5NjwvWWVh
cj48UmVjTnVtPjM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM8L3JlYy1udW1iZXI+PGZv
cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVz
cGF4eHhyZnpmc3ciPjM8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FB
QXB2TUNFIj41MzI8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+Njwv
cmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkJhY2htYW4sIEx5bGUgRi48
L2F1dGhvcj48YXV0aG9yPlBhbG1lciwgQWRyaWFuLiBTLjwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5MYW5ndWFnZSB0ZXN0aW5nIGluIHByYWN0aWNlOiBE
ZXNpZ25pbmcgYW5kIGRldmVsb3BpbmcgdXNlZnVsIGxhbmd1YWdlIHRlc3RzPC90aXRsZT48L3Rp
dGxlcz48ZGF0ZXM+PHllYXI+MTk5NjwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+T3hmb3Jk
PC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5PeGZvcmQgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlz
aGVyPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5HcmFiZTwvQXV0
aG9yPjxZZWFyPjE5OTY8L1llYXI+PFJlY051bT43MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1i
ZXI+NzA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJm
MGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjcwPC9rZXk+PGtleSBhcHA9IkVOV2Vi
IiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTg1PC9rZXk+PC9mb3JlaWduLWtleXM+PHJl
Zi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5XaWxsaWFtIEdyYWJlPC9hdXRob3I+PGF1dGhvcj5Sb2JlcnQgQi4gS2FwbGFuPC9hdXRo
b3I+PC9hdXRob3JzPjxzZWNvbmRhcnktYXV0aG9ycz48YXV0aG9yPkNocmlzdG9waGVyIE4uIENh
bmRsaW48L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+
PHRpdGxlPlRoZW9yeSAmYW1wOyBwcmFjdGljZSBvZiB3cml0aW5nOiBBbiBhcHBsaWVkIGxpbmd1
aXN0aWMgcGVyc3BlY3RpdmU8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBwbGllZCBMaW5ndWlz
dGljcyBhbmQgTGFuZ3VhZ2UgU3R1ZHk8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+
NDg3PC9wYWdlcz48a2V5d29yZHM+PGtleXdvcmQ+d3JpdGluZyBwcm9jZXNzLCB0aGVvcnkgb2Yg
d3JpdGluZywgc2Vjb25kIGxhbmd1YWdlLCBjb250cmFzdGl2ZSByaGV0b3JpYzwva2V5d29yZD48
L2tleXdvcmRzPjxkYXRlcz48eWVhcj4xOTk2PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5O
ZXcgWW9yazwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+QWRkaXNvbiBXZXNsZXkgTG9uZ21hbjwv
cHVibGlzaGVyPjxpc2JuPjAtNTgyLTU1MzgzLTA8L2lzYm4+PGNhbGwtbnVtPlBFMTQwNCBHNjUg
MTk5NjwvY2FsbC1udW0+PHVybHM+PC91cmxzPjxyZXNlYXJjaC1ub3Rlcz5Db250YWlucyBrZXkg
d3JpdGluZyBtb2RlbCBmb3IgTDIgd3JpdGluZzwvcmVzZWFyY2gtbm90ZXM+PGxhbmd1YWdlPkVu
Z2xpc2g8L2xhbmd1YWdlPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XZWlnbGU8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+NTwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQmFjaG1hbiwgMTk5MDsgQmFjaG1hbiAmYW1w
OyBQYWxtZXIsIDE5OTY7IEdyYWJlICZhbXA7IEthcGxhbiwgMTk5NjsgV2VpZ2xlLCAyMDAyKTwv
RGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj41PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl
eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6
ZnN3Ij41PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+
NTM0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XZWlnbGUsIFNhcmEgQ3VzaGluZzwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Bc3Nlc3Npbmcgd3Jp
dGluZzwvdGl0bGU+PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIwMDI8L3llYXI+PC9kYXRlcz48cHVi
LWxvY2F0aW9uPkNhbWJyaWRnZTwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+Q2FtYnJpZGdlIFVu
aXZlcnNpdHkgUHJlc3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD
aXRlPjxBdXRob3I+QmFjaG1hbjwvQXV0aG9yPjxZZWFyPjE5OTA8L1llYXI+PFJlY051bT4yPC9S
ZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl
eSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4y
PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTMxPC9r
ZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5CYWNobWFuLCBMeWxlIEYuPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkZ1bmRhbWVudGFsIGNvbnNpZGVyYXRp
b25zIGluIGxhbmd1YWdlIHRlc3RpbmcuPC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MTk5
MDwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+T3hmb3JkPC9wdWItbG9jYXRpb24+PHB1Ymxp
c2hlcj5PeGZvcmQgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjx1cmxzPjwvdXJscz48L3Jl
Y29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5CYWNobWFuPC9BdXRob3I+PFllYXI+MTk5NjwvWWVh
cj48UmVjTnVtPjM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM8L3JlYy1udW1iZXI+PGZv
cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVz
cGF4eHhyZnpmc3ciPjM8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FB
QXB2TUNFIj41MzI8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+Njwv
cmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkJhY2htYW4sIEx5bGUgRi48
L2F1dGhvcj48YXV0aG9yPlBhbG1lciwgQWRyaWFuLiBTLjwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5MYW5ndWFnZSB0ZXN0aW5nIGluIHByYWN0aWNlOiBE
ZXNpZ25pbmcgYW5kIGRldmVsb3BpbmcgdXNlZnVsIGxhbmd1YWdlIHRlc3RzPC90aXRsZT48L3Rp
dGxlcz48ZGF0ZXM+PHllYXI+MTk5NjwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+T3hmb3Jk
PC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5PeGZvcmQgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlz
aGVyPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5HcmFiZTwvQXV0
aG9yPjxZZWFyPjE5OTY8L1llYXI+PFJlY051bT43MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1i
ZXI+NzA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJm
MGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjcwPC9rZXk+PGtleSBhcHA9IkVOV2Vi
IiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTg1PC9rZXk+PC9mb3JlaWduLWtleXM+PHJl
Zi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5XaWxsaWFtIEdyYWJlPC9hdXRob3I+PGF1dGhvcj5Sb2JlcnQgQi4gS2FwbGFuPC9hdXRo
b3I+PC9hdXRob3JzPjxzZWNvbmRhcnktYXV0aG9ycz48YXV0aG9yPkNocmlzdG9waGVyIE4uIENh
bmRsaW48L2F1dGhvcj48L3NlY29uZGFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+
PHRpdGxlPlRoZW9yeSAmYW1wOyBwcmFjdGljZSBvZiB3cml0aW5nOiBBbiBhcHBsaWVkIGxpbmd1
aXN0aWMgcGVyc3BlY3RpdmU8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBwbGllZCBMaW5ndWlz
dGljcyBhbmQgTGFuZ3VhZ2UgU3R1ZHk8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+
NDg3PC9wYWdlcz48a2V5d29yZHM+PGtleXdvcmQ+d3JpdGluZyBwcm9jZXNzLCB0aGVvcnkgb2Yg
d3JpdGluZywgc2Vjb25kIGxhbmd1YWdlLCBjb250cmFzdGl2ZSByaGV0b3JpYzwva2V5d29yZD48
L2tleXdvcmRzPjxkYXRlcz48eWVhcj4xOTk2PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5O
ZXcgWW9yazwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+QWRkaXNvbiBXZXNsZXkgTG9uZ21hbjwv
cHVibGlzaGVyPjxpc2JuPjAtNTgyLTU1MzgzLTA8L2lzYm4+PGNhbGwtbnVtPlBFMTQwNCBHNjUg
MTk5NjwvY2FsbC1udW0+PHVybHM+PC91cmxzPjxyZXNlYXJjaC1ub3Rlcz5Db250YWlucyBrZXkg
d3JpdGluZyBtb2RlbCBmb3IgTDIgd3JpdGluZzwvcmVzZWFyY2gtbm90ZXM+PGxhbmd1YWdlPkVu
Z2xpc2g8L2xhbmd1YWdlPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE.DATA (Bachman, 1990; Bachman & Palmer, 1996; Grabe & Kaplan, 1996; Weigle, 2002). In both L1 and L2 writing models, knowledge pertaining to content, form, and process as well as theoretical language knowledge are essential to produce texts. An important component of this language knowledge, in particular in the language testing literature, is grammatical knowledge because it is seen as indispensable in accomplishing a task in a L2 ADDIN EN.CITE <EndNote><Cite><Author>Bachman</Author><Year>1996</Year><RecNum>3</RecNum><DisplayText>(Bachman, 1990; Bachman & Palmer, 1996; Weigle, 2002)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">3</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">532</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Bachman, Lyle F.</author><author>Palmer, Adrian. S.</author></authors></contributors><titles><title>Language testing in practice: Designing and developing useful language tests</title></titles><dates><year>1996</year></dates><pub-location>Oxford</pub-location><publisher>Oxford University Press</publisher><urls></urls></record></Cite><Cite><Author>Bachman</Author><Year>1990</Year><RecNum>2</RecNum><record><rec-number>2</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">2</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">531</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Bachman, Lyle F.</author></authors></contributors><titles><title>Fundamental considerations in language testing.</title></titles><dates><year>1990</year></dates><pub-location>Oxford</pub-location><publisher>Oxford University Press</publisher><urls></urls></record></Cite><Cite><Author>Weigle</Author><Year>2002</Year><RecNum>5</RecNum><record><rec-number>5</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">5</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">534</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Weigle, Sara Cushing</author></authors></contributors><titles><title>Assessing writing</title></titles><dates><year>2002</year></dates><pub-location>Cambridge</pub-location><publisher>Cambridge University Press</publisher><urls></urls></record></Cite></EndNote>(Bachman, 1990; Bachman & Palmer, 1996; Weigle, 2002). There are various models of grammatical or language ability PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XZWlnbGU8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+NTwvUmVjTnVtPjxQcmVmaXg+ZS5nLmAsIDwvUHJlZml4PjxEaXNwbGF5VGV4dD4oZS5n
LiwgQmFjaG1hbiwgMTk5MDsgQmFjaG1hbiAmYW1wOyBQYWxtZXIsIDE5OTY7IENhbmFsZSAmYW1w
OyBTd2FpbiwgMTk4MDsgUHVycHVyYSwgMjAwNDsgV2VpZ2xlLCAyMDAyKTwvRGlzcGxheVRleHQ+
PHJlY29yZD48cmVjLW51bWJlcj41PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9
IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij41PC9rZXk+
PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM0PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRv
cnM+PGF1dGhvcnM+PGF1dGhvcj5XZWlnbGUsIFNhcmEgQ3VzaGluZzwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Bc3Nlc3Npbmcgd3JpdGluZzwvdGl0bGU+
PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIwMDI8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNh
bWJyaWRnZTwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJl
c3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+
UHVycHVyYTwvQXV0aG9yPjxZZWFyPjIwMDQ8L1llYXI+PFJlY051bT42PC9SZWNOdW0+PHJlY29y
ZD48cmVjLW51bWJlcj42PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij42PC9rZXk+PGtleSBh
cHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM1PC9rZXk+PC9mb3JlaWdu
LWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1
dGhvcnM+PGF1dGhvcj5QdXJwdXJhLCBKYW1lcyBFLjwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25k
YXJ5LWF1dGhvcnM+PGF1dGhvcj5BbGRlcnNvbiwgSi4gQ2hhcmxlczwvYXV0aG9yPjxhdXRob3I+
QmFjaG1hbiwgTHlsZSBGLjwvYXV0aG9yPjwvc2Vjb25kYXJ5LWF1dGhvcnM+PC9jb250cmlidXRv
cnM+PHRpdGxlcz48dGl0bGU+QXNzZXNzaW5nIGdyYW1tYXI8L3RpdGxlPjxzZWNvbmRhcnktdGl0
bGU+Q2FtYnJpZGdlIExhbmd1YWdlIEFzc2Vzc21lbnQgU2VyaWVzPC9zZWNvbmRhcnktdGl0bGU+
PC90aXRsZXM+PHBhZ2VzPjMwNTwvcGFnZXM+PGtleXdvcmRzPjxrZXl3b3JkPmFzc2Vzc21lbnQs
IGdyYW1tYXI8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwNDwveWVhcj48L2Rh
dGVzPjxwdWItbG9jYXRpb24+Q2FtYnJpZGdlPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5DYW1i
cmlkZ2UgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjxpc2JuPjAtNTIxLTgwMjgxLTQ8L2lz
Ym4+PGNhbGwtbnVtPlAxNTEgUDg3IDIwMDQ8L2NhbGwtbnVtPjx1cmxzPjwvdXJscz48L3JlY29y
ZD48L0NpdGU+PENpdGU+PEF1dGhvcj5DYW5hbGU8L0F1dGhvcj48WWVhcj4xOTgwPC9ZZWFyPjxS
ZWNOdW0+MTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTwvcmVjLW51bWJlcj48Zm9yZWln
bi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4
eHJmemZzdyI+MTwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZN
Q0UiPjUzMDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGlj
bGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Q2FuYWxlLCBN
LjwvYXV0aG9yPjxhdXRob3I+U3dhaW4sIE0uPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0
b3JzPjx0aXRsZXM+PHRpdGxlPlRoZW9yZXRpY2FsIGJhc2VzIG9mIGNvbW11bmljYXRpdmUgYXBw
cm9hY2hlcyB0byBzZWNvbmQgbGFuZ3VhZ2UgdGVhY2hpbmcgYW5kIHRlc3Rpbmc8L3RpdGxlPjxz
ZWNvbmRhcnktdGl0bGU+QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0
bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0
bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz4xLTQ3PC9wYWdlcz48dm9sdW1lPjE8L3ZvbHVtZT48bnVt
YmVyPjE8L251bWJlcj48ZGF0ZXM+PHllYXI+MTk4MDwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJs
cz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA5My9hcHBsaW4vSS4xLjEgPC9lbGVjdHJv
bmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5CYWNobWFuPC9B
dXRob3I+PFllYXI+MTk5NjwvWWVhcj48UmVjTnVtPjM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVt
YmVyPjM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJm
MGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjM8L2tleT48a2V5IGFwcD0iRU5XZWIi
IGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41MzI8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVm
LXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkJhY2htYW4sIEx5bGUgRi48L2F1dGhvcj48YXV0aG9yPlBhbG1lciwgQWRyaWFuLiBTLjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5MYW5ndWFnZSB0
ZXN0aW5nIGluIHByYWN0aWNlOiBEZXNpZ25pbmcgYW5kIGRldmVsb3BpbmcgdXNlZnVsIGxhbmd1
YWdlIHRlc3RzPC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MTk5NjwveWVhcj48L2RhdGVz
PjxwdWItbG9jYXRpb24+T3hmb3JkPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5PeGZvcmQgVW5p
dmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENp
dGU+PEF1dGhvcj5CYWNobWFuPC9BdXRob3I+PFllYXI+MTk5MDwvWWVhcj48UmVjTnVtPjI8L1Jl
Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjI8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjI8
L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41MzE8L2tl
eT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRy
aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkJhY2htYW4sIEx5bGUgRi48L2F1dGhvcj48L2F1dGhv
cnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+RnVuZGFtZW50YWwgY29uc2lkZXJhdGlv
bnMgaW4gbGFuZ3VhZ2UgdGVzdGluZy48L3RpdGxlPjwvdGl0bGVzPjxkYXRlcz48eWVhcj4xOTkw
PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5PeGZvcmQ8L3B1Yi1sb2NhdGlvbj48cHVibGlz
aGVyPk94Zm9yZCBVbml2ZXJzaXR5IFByZXNzPC9wdWJsaXNoZXI+PHVybHM+PC91cmxzPjwvcmVj
b3JkPjwvQ2l0ZT48L0VuZE5vdGU+AG==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XZWlnbGU8L0F1dGhvcj48WWVhcj4yMDAyPC9ZZWFyPjxS
ZWNOdW0+NTwvUmVjTnVtPjxQcmVmaXg+ZS5nLmAsIDwvUHJlZml4PjxEaXNwbGF5VGV4dD4oZS5n
LiwgQmFjaG1hbiwgMTk5MDsgQmFjaG1hbiAmYW1wOyBQYWxtZXIsIDE5OTY7IENhbmFsZSAmYW1w
OyBTd2FpbiwgMTk4MDsgUHVycHVyYSwgMjAwNDsgV2VpZ2xlLCAyMDAyKTwvRGlzcGxheVRleHQ+
PHJlY29yZD48cmVjLW51bWJlcj41PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9
IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij41PC9rZXk+
PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM0PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRv
cnM+PGF1dGhvcnM+PGF1dGhvcj5XZWlnbGUsIFNhcmEgQ3VzaGluZzwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Bc3Nlc3Npbmcgd3JpdGluZzwvdGl0bGU+
PC90aXRsZXM+PGRhdGVzPjx5ZWFyPjIwMDI8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkNh
bWJyaWRnZTwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+Q2FtYnJpZGdlIFVuaXZlcnNpdHkgUHJl
c3M8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+
UHVycHVyYTwvQXV0aG9yPjxZZWFyPjIwMDQ8L1llYXI+PFJlY051bT42PC9SZWNOdW0+PHJlY29y
ZD48cmVjLW51bWJlcj42PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij42PC9rZXk+PGtleSBh
cHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM1PC9rZXk+PC9mb3JlaWdu
LWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1
dGhvcnM+PGF1dGhvcj5QdXJwdXJhLCBKYW1lcyBFLjwvYXV0aG9yPjwvYXV0aG9ycz48c2Vjb25k
YXJ5LWF1dGhvcnM+PGF1dGhvcj5BbGRlcnNvbiwgSi4gQ2hhcmxlczwvYXV0aG9yPjxhdXRob3I+
QmFjaG1hbiwgTHlsZSBGLjwvYXV0aG9yPjwvc2Vjb25kYXJ5LWF1dGhvcnM+PC9jb250cmlidXRv
cnM+PHRpdGxlcz48dGl0bGU+QXNzZXNzaW5nIGdyYW1tYXI8L3RpdGxlPjxzZWNvbmRhcnktdGl0
bGU+Q2FtYnJpZGdlIExhbmd1YWdlIEFzc2Vzc21lbnQgU2VyaWVzPC9zZWNvbmRhcnktdGl0bGU+
PC90aXRsZXM+PHBhZ2VzPjMwNTwvcGFnZXM+PGtleXdvcmRzPjxrZXl3b3JkPmFzc2Vzc21lbnQs
IGdyYW1tYXI8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwNDwveWVhcj48L2Rh
dGVzPjxwdWItbG9jYXRpb24+Q2FtYnJpZGdlPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5DYW1i
cmlkZ2UgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjxpc2JuPjAtNTIxLTgwMjgxLTQ8L2lz
Ym4+PGNhbGwtbnVtPlAxNTEgUDg3IDIwMDQ8L2NhbGwtbnVtPjx1cmxzPjwvdXJscz48L3JlY29y
ZD48L0NpdGU+PENpdGU+PEF1dGhvcj5DYW5hbGU8L0F1dGhvcj48WWVhcj4xOTgwPC9ZZWFyPjxS
ZWNOdW0+MTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTwvcmVjLW51bWJlcj48Zm9yZWln
bi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4
eHJmemZzdyI+MTwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZN
Q0UiPjUzMDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGlj
bGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Q2FuYWxlLCBN
LjwvYXV0aG9yPjxhdXRob3I+U3dhaW4sIE0uPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0
b3JzPjx0aXRsZXM+PHRpdGxlPlRoZW9yZXRpY2FsIGJhc2VzIG9mIGNvbW11bmljYXRpdmUgYXBw
cm9hY2hlcyB0byBzZWNvbmQgbGFuZ3VhZ2UgdGVhY2hpbmcgYW5kIHRlc3Rpbmc8L3RpdGxlPjxz
ZWNvbmRhcnktdGl0bGU+QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0
bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0
bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz4xLTQ3PC9wYWdlcz48dm9sdW1lPjE8L3ZvbHVtZT48bnVt
YmVyPjE8L251bWJlcj48ZGF0ZXM+PHllYXI+MTk4MDwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJs
cz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA5My9hcHBsaW4vSS4xLjEgPC9lbGVjdHJv
bmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5CYWNobWFuPC9B
dXRob3I+PFllYXI+MTk5NjwvWWVhcj48UmVjTnVtPjM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVt
YmVyPjM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJm
MGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjM8L2tleT48a2V5IGFwcD0iRU5XZWIi
IGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41MzI8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVm
LXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkJhY2htYW4sIEx5bGUgRi48L2F1dGhvcj48YXV0aG9yPlBhbG1lciwgQWRyaWFuLiBTLjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5MYW5ndWFnZSB0
ZXN0aW5nIGluIHByYWN0aWNlOiBEZXNpZ25pbmcgYW5kIGRldmVsb3BpbmcgdXNlZnVsIGxhbmd1
YWdlIHRlc3RzPC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MTk5NjwveWVhcj48L2RhdGVz
PjxwdWItbG9jYXRpb24+T3hmb3JkPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5PeGZvcmQgVW5p
dmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENp
dGU+PEF1dGhvcj5CYWNobWFuPC9BdXRob3I+PFllYXI+MTk5MDwvWWVhcj48UmVjTnVtPjI8L1Jl
Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjI8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjI8
L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41MzE8L2tl
eT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRy
aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkJhY2htYW4sIEx5bGUgRi48L2F1dGhvcj48L2F1dGhv
cnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+RnVuZGFtZW50YWwgY29uc2lkZXJhdGlv
bnMgaW4gbGFuZ3VhZ2UgdGVzdGluZy48L3RpdGxlPjwvdGl0bGVzPjxkYXRlcz48eWVhcj4xOTkw
PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5PeGZvcmQ8L3B1Yi1sb2NhdGlvbj48cHVibGlz
aGVyPk94Zm9yZCBVbml2ZXJzaXR5IFByZXNzPC9wdWJsaXNoZXI+PHVybHM+PC91cmxzPjwvcmVj
b3JkPjwvQ2l0ZT48L0VuZE5vdGU+AG==
ADDIN EN.CITE.DATA (e.g., Bachman, 1990; Bachman & Palmer, 1996; Canale & Swain, 1980; Purpura, 2004; Weigle, 2002), and while the format of these models differs, grammatical ability in all these models comprises the learner’s ability to use and apply theoretical grammatical knowledge accurately and meaningfully in language use situations. The grammar knowledge to be applied relates to graphology, lexicon, and a range of morphosyntactic forms and structures. The question is, however, whether teachers’ construct of grammatical ability resembles the theoretical construct definitions such models put forward.Indicators of Grammatical Ability in L2 Writing ResearchThe goal of the current study is to examine the construct of grammatical ability that is assessed in L2 academic writing courses. In order to examine this construct, an analysis of L2 texts is crucial because the texts, along with the rater who assigns the grade and the rating scale used by the rater, influence the score or grade for this performance ADDIN EN.CITE <EndNote><Cite><Author>Upshur</Author><Year>1999</Year><RecNum>235</RecNum><DisplayText>(McNamara, 1996; Upshur & Turner, 1999)</DisplayText><record><rec-number>235</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">235</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">738</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Upshur, John A.</author><author>Turner, Carolyn E.</author></authors></contributors><titles><title>Systematic effects in the rating of second-language speaking ability: Test method and learner discourse</title><secondary-title>Language Testing</secondary-title></titles><periodical><full-title>Language Testing</full-title></periodical><pages>82-111</pages><volume>16</volume><number>1</number><dates><year>1999</year><pub-dates><date>January 1, 1999</date></pub-dates></dates><urls><related-urls><url> app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">4</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">533</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>McNamara, Timothy F.</author></authors></contributors><titles><title>Measuring second language performance</title></titles><dates><year>1996</year></dates><pub-location>New York</pub-location><publisher>Longman</publisher><urls></urls></record></Cite></EndNote>(McNamara, 1996; Upshur & Turner, 1999).Both grammatical accuracy and grammatical complexity are key constructs in research studies dealing with the assessment of grammatical ability in L2 writing and analyzing L2 texts for indicators of grammatical ability. Many studies rely on operationalizations of grammatical accuracy in conjunction with grammatical complexity to obtain more comprehensive indicators of test takers’ language proficiency or level of linguistic performance PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Jd2FzaGl0YTwvQXV0aG9yPjxZZWFyPjIwMDg8L1llYXI+
PFJlY051bT4yMTY8L1JlY051bT48RGlzcGxheVRleHQ+KGUuZy4sIEN1bW1pbmcgZXQgYWwuLCAy
MDA1OyBFbGxpcyAmYW1wOyBZdWFuLCAyMDA0OyBIb3VzZW4gJmFtcDsgS3Vpa2VuLCAyMDA5Yjsg
SXdhc2hpdGEsIEJyb3duLCBNY05hbWFyYSwgJmFtcDsgTyZhcG9zO0hhZ2FuLCAyMDA4OyBLdWlr
ZW4gJmFtcDsgVmVkZGVyLCAyMDA4OyBMaSwgMjAwMDsgU3RvcmNoLCAyMDA5OyBXaWdnbGVzd29y
dGggJmFtcDsgU3RvcmNoLCAyMDA5OyBXb2xmZS1RdWludGVybywgSW5hZ2FraSwgJmFtcDsgS2lt
LCAxOTk4KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4yMTY8L3JlYy1udW1iZXI+
PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5
cnVzcGF4eHhyZnpmc3ciPjIxNjwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0
cWdjQUFBcHZNQ0UiPjcxOTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3Vy
bmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+
SXdhc2hpdGEsIE5vcmlrbzwvYXV0aG9yPjxhdXRob3I+QnJvd24sIEFubmllPC9hdXRob3I+PGF1
dGhvcj5NY05hbWFyYSwgVGltPC9hdXRob3I+PGF1dGhvcj5PJmFwb3M7SGFnYW4sIFNhbGx5PC9h
dXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc2Vzc2VkIGxl
dmVscyBvZiBzZWNvbmQgbGFuZ3VhZ2Ugc3BlYWtpbmcgcHJvZmljaWVuY3k6IEhvdyBkaXN0aW5j
dD88L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5
LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3Rp
Y3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz4yNC00OTwvcGFnZXM+PHZvbHVtZT4y
OTwvdm9sdW1lPjxudW1iZXI+MTwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDA4PC95ZWFyPjxwdWIt
ZGF0ZXM+PGRhdGU+TWFyY2ggMSwgMjAwODwvZGF0ZT48L3B1Yi1kYXRlcz48L2RhdGVzPjx1cmxz
PjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vYXBwbGlqLm94Zm9yZGpvdXJuYWxzLm9yZy9jZ2kv
Y29udGVudC9hYnN0cmFjdC8yOS8xLzI0IDwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxl
Y3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA5My9hcHBsaW4vYW1tMDE3PC9lbGVjdHJvbmljLXJl
c291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Ib3VzZW48L0F1dGhvcj48
WWVhcj4yMDA5PC9ZZWFyPjxSZWNOdW0+NDEzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40
MTM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5
NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQxMzwva2V5PjwvZm9yZWlnbi1rZXlzPjxy
ZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3Jz
PjxhdXRob3JzPjxhdXRob3I+SG91c2VuLCBBbGV4PC9hdXRob3I+PGF1dGhvcj5LdWlrZW4sIEZv
bGtlcnQ8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+Q29t
cGxleGl0eSwgYWNjdXJhY3ksIGFuZCBmbHVlbmN5IGluIHNlY29uZCBsYW5ndWFnZSBhY3F1aXNp
dGlvbiBbU3BlY2lhbCBpc3N1ZV08L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBwbGllZCBMaW5n
dWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxl
PkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz40NjEg
LSA2MjYgPC9wYWdlcz48dm9sdW1lPjMwPC92b2x1bWU+PG51bWJlcj40PC9udW1iZXI+PGRhdGVz
Pjx5ZWFyPjIwMDk8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PGN1c3RvbTI+QXBwbGllZCBM
aW5ndWlzdGljczwvY3VzdG9tMj48Y3VzdG9tMz5Ib3VzZW4sIEEuLCAmYW1wOyBLdWlrZW4sIEYu
IChFZHMuKTwvY3VzdG9tMz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5DdW1taW5nPC9B
dXRob3I+PFllYXI+MjAwNTwvWWVhcj48UmVjTnVtPjI2NzwvUmVjTnVtPjxQcmVmaXg+ZS5nLmAs
IDwvUHJlZml4PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjY3PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl
eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6
ZnN3Ij4yNjc8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNF
Ij43Njk8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xl
Ij4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkN1bW1pbmcsIEFs
aXN0ZXI8L2F1dGhvcj48YXV0aG9yPkthbnRvciwgUm9iZXJ0PC9hdXRob3I+PGF1dGhvcj5CYWJh
LCBLeW9rbzwvYXV0aG9yPjxhdXRob3I+RXJkb3N5LCBVc21hbjwvYXV0aG9yPjxhdXRob3I+RW91
YW56b3VpLCBLZWFucmU8L2F1dGhvcj48YXV0aG9yPkphbWVzLCBNYXJrPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkRpZmZlcmVuY2VzIGluIHdyaXR0ZW4g
ZGlzY291cnNlIGluIGluZGVwZW5kZW50IGFuZCBpbnRlZ3JhdGVkIHByb3RvdHlwZSB0YXNrcyBm
b3IgbmV4dCBnZW5lcmF0aW9uIFRPRUZMPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFzc2Vzc2lu
ZyBXcml0aW5nPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0
bGU+QXNzZXNzaW5nIFdyaXRpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz41LTQz
PC9wYWdlcz48dm9sdW1lPjEwPC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGtleXdvcmRzPjxr
ZXl3b3JkPldyaXRpbmcgYXNzZXNzbWVudDwva2V5d29yZD48a2V5d29yZD5UT0VGTDwva2V5d29y
ZD48a2V5d29yZD5JbnRlZ3JhdGVkIHRhc2tzPC9rZXl3b3JkPjxrZXl3b3JkPkRpc2NvdXJzZSBh
bmFseXNpczwva2V5d29yZD48a2V5d29yZD5FU0wgd3JpdGluZzwva2V5d29yZD48L2tleXdvcmRz
PjxkYXRlcz48eWVhcj4yMDA1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTA3NS0yOTM1PC9pc2JuPjx1
cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vd3d3LnNjaWVuY2VkaXJlY3QuY29tL3NjaWVu
Y2UvYXJ0aWNsZS9CNlZUOC00RlhOUlBDLTEvMi8wNGE3MTk1NzRhMGY2OGFiZGUxMTQzZmZkYjRk
ZWRiNzwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+
MTAuMTAxNi9qLmFzdy4yMDA1LjAyLjAwMTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNv
cmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+RWxsaXM8L0F1dGhvcj48WWVhcj4yMDA0PC9ZZWFyPjxS
ZWNOdW0+NjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjYxPC9yZWMtbnVtYmVyPjxmb3Jl
aWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3Bh
eHh4cmZ6ZnN3Ij42MTwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFB
cHZNQ0UiPjU3Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFy
dGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+RWxsaXMs
IFJvZDwvYXV0aG9yPjxhdXRob3I+WXVhbiwgRmFuZ3l1YW48L2F1dGhvcj48L2F1dGhvcnM+PC9j
b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VGhlIGVmZmVjdHMgb2YgcGxhbm5pbmcgb24gZmx1
ZW5jeSwgY29tcGxleGl0eSwgYW5kIGFjY3VyYWN5IGluIHNlY29uZCBsYW5ndWFnZSBuYXJyYXRp
dmUgd3JpdGluZzwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TdHVkaWVzIGluIFNlY29uZCBMYW5n
dWFnZSBBY3F1aXNpdGlvbjwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxm
dWxsLXRpdGxlPlN0dWRpZXMgaW4gU2Vjb25kIExhbmd1YWdlIEFjcXVpc2l0aW9uPC9mdWxsLXRp
dGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NTktODQ8L3BhZ2VzPjx2b2x1bWU+MjY8L3ZvbHVtZT48
bnVtYmVyPjE8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwNDwveWVhcj48L2RhdGVzPjx1cmxzPjwv
dXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTAxNy9TMDI3MjI2MzEwNDAyNjEzMDwv
ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+S3Vp
a2VuPC9BdXRob3I+PFllYXI+MjAwODwvWWVhcj48UmVjTnVtPjIxMTwvUmVjTnVtPjxyZWNvcmQ+
PHJlYy1udW1iZXI+MjExPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yMTE8L2tleT48a2V5
IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43MTQ8L2tleT48L2ZvcmVp
Z24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNv
bnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkt1aWtlbiwgRm9sa2VydDwvYXV0aG9yPjxhdXRo
b3I+VmVkZGVyLCBJbmVrZTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5Db2duaXRpdmUgdGFzayBjb21wbGV4aXR5IGFuZCB3cml0dGVuIG91dHB1dCBpbiBJ
dGFsaWFuIGFuZCBGcmVuY2ggYXMgYSBmb3JlaWduIGxhbmd1YWdlPC90aXRsZT48c2Vjb25kYXJ5
LXRpdGxlPkpvdXJuYWwgb2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNlY29uZCBMYW5n
dWFnZSBXcml0aW5nPC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NDgtNjA8L3BhZ2Vz
Pjx2b2x1bWU+MTc8L3ZvbHVtZT48bnVtYmVyPjE8L251bWJlcj48a2V5d29yZHM+PGtleXdvcmQ+
QWNjdXJhY3k8L2tleXdvcmQ+PGtleXdvcmQ+QXR0ZW50aW9uPC9rZXl3b3JkPjxrZXl3b3JkPkNv
Z25pdGlvbiBIeXBvdGhlc2lzPC9rZXl3b3JkPjxrZXl3b3JkPkNvZ25pdGl2ZSB0YXNrIGNvbXBs
ZXhpdHk8L2tleXdvcmQ+PGtleXdvcmQ+TGV4aWNhbCB2YXJpYXRpb248L2tleXdvcmQ+PGtleXdv
cmQ+TGltaXRlZCBBdHRlbnRpb25hbCBDYXBhY2l0eSBNb2RlbDwva2V5d29yZD48a2V5d29yZD5N
dWx0aXBsZSBBdHRlbnRpb25hbCBSZXNvdXJjZXMgTW9kZWw8L2tleXdvcmQ+PGtleXdvcmQ+UHJv
ZmljaWVuY3kgbGV2ZWw8L2tleXdvcmQ+PGtleXdvcmQ+U3ludGFjdGljIGNvbXBsZXhpdHk8L2tl
eXdvcmQ+PGtleXdvcmQ+V3JpdGluZyBwZXJmb3JtYW5jZTwva2V5d29yZD48L2tleXdvcmRzPjxk
YXRlcz48eWVhcj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0
dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20vc2NpZW5jZS9hcnRpY2xlL0I2VzVGLTRSMTFLRkMt
My8xL2NlNzNlNGI0NWQ0MjY5NzY0OTc4MjE4MWUwMTRiYTNhIDwvdXJsPjwvcmVsYXRlZC11cmxz
PjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTAxNi9qLmpzbHcuMjAwNy4wOC4w
MDMgPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhv
cj5MaTwvQXV0aG9yPjxZZWFyPjIwMDA8L1llYXI+PFJlY051bT4xNjI8L1JlY051bT48cmVjb3Jk
PjxyZWMtbnVtYmVyPjE2MjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIg
ZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+MTYyPC9rZXk+PGtl
eSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NjY5PC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5MaSwgWWlsaTwvYXV0aG9yPjwvYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5MaW5ndWlzdGljIGNoYXJhY3RlcmlzdGljcyBv
ZiBFU0wgd3JpdGluZyBpbiB0YXNrLWJhc2VkIGUtbWFpbCBhY3Rpdml0aWVzPC90aXRsZT48c2Vj
b25kYXJ5LXRpdGxlPlN5c3RlbTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2Fs
PjxmdWxsLXRpdGxlPlN5c3RlbTwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjIyOS0y
NDU8L3BhZ2VzPjx2b2x1bWU+Mjg8L3ZvbHVtZT48bnVtYmVyPjI8L251bWJlcj48ZGF0ZXM+PHll
YXI+MjAwMDwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1u
dW0+MTAuMTAxNi9TMDM0Ni0yNTFYKDAwKTAwMDA5LTkgPC9lbGVjdHJvbmljLXJlc291cmNlLW51
bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5TdG9yY2g8L0F1dGhvcj48WWVhcj4yMDA5
PC9ZZWFyPjxSZWNOdW0+MzgyPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zODI8L3JlYy1u
dW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2Vh
djBwcDk5cnVzcGF4eHhyZnpmc3ciPjM4Mjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu
YW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz
PjxhdXRob3I+U3RvcmNoLCBOZW9teTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48
dGl0bGVzPjx0aXRsZT5UaGUgaW1wYWN0IG9mIHN0dWR5aW5nIGluIGEgc2Vjb25kIGxhbmd1YWdl
IChMMikgbWVkaXVtIHVuaXZlcnNpdHkgb24gdGhlIGRldmVsb3BtZW50IG9mIEwyIHdyaXRpbmc8
L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBTZWNvbmQgTGFuZ3VhZ2UgV3JpdGlu
Zzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkpvdXJu
YWwgb2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxw
YWdlcz4xMDMtMTE4PC9wYWdlcz48dm9sdW1lPjE4PC92b2x1bWU+PG51bWJlcj4yPC9udW1iZXI+
PGtleXdvcmRzPjxrZXl3b3JkPlNlY29uZCBsYW5ndWFnZSB3cml0aW5nIGRldmVsb3BtZW50PC9r
ZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PC9kYXRlcz48aXNibj4x
MDYwLTM3NDM8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5j
ZWRpcmVjdC5jb20vc2NpZW5jZS9hcnRpY2xlL0I2VzVGLTRWWENHNUstMS8yLzI5NWUxYWM1MzU0
NDI5MjdjZDA5ODk5NjM1OTkyNzM1PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJv
bmljLXJlc291cmNlLW51bT4xMC4xMDE2L2ouanNsdy4yMDA5LjAyLjAwMzwvZWxlY3Ryb25pYy1y
ZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V2lnZ2xlc3dvcnRoPC9B
dXRob3I+PFllYXI+MjAwOTwvWWVhcj48UmVjTnVtPjI2NjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1u
dW1iZXI+MjY2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
ZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yNjY8L2tleT48a2V5IGFwcD0i
RU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43Njg8L2tleT48L2ZvcmVpZ24ta2V5
cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1
dG9ycz48YXV0aG9ycz48YXV0aG9yPldpZ2dsZXN3b3J0aCwgR2lsbGlhbjwvYXV0aG9yPjxhdXRo
b3I+U3RvcmNoLCBOZW9teTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5QYWlyIHZlcnN1cyBpbmRpdmlkdWFsIHdyaXRpbmc6IEVmZmVjdHMgb24gZmx1ZW5j
eSwgY29tcGxleGl0eSBhbmQgYWNjdXJhY3k8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+TGFuZ3Vh
Z2UgVGVzdGluZzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRp
dGxlPkxhbmd1YWdlIFRlc3Rpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz40NDUt
NDY2PC9wYWdlcz48dm9sdW1lPjI2PC92b2x1bWU+PG51bWJlcj4zPC9udW1iZXI+PGRhdGVzPjx5
ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5KdWx5IDEsIDIwMDk8L2RhdGU+PC9wdWIt
ZGF0ZXM+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2x0ai5zYWdlcHVi
LmNvbS9jZ2kvY29udGVudC9hYnN0cmFjdC8yNi8zLzQ0NTwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTE3Ny8wMjY1NTMyMjA5MTA0NjcwPC9l
bGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Xb2xm
ZS1RdWludGVybzwvQXV0aG9yPjxZZWFyPjE5OTg8L1llYXI+PFJlY051bT4xMzwvUmVjTnVtPjxy
ZWNvcmQ+PHJlYy1udW1iZXI+MTM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i
RU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjEzPC9rZXk+
PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM4PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRv
cnM+PGF1dGhvcnM+PGF1dGhvcj5Xb2xmZS1RdWludGVybywgS2F0ZTwvYXV0aG9yPjxhdXRob3I+
SW5hZ2FraSwgU2h1bmppPC9hdXRob3I+PGF1dGhvcj5LaW0sIEhhZS1Zb3VuZzwvYXV0aG9yPjwv
YXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5TZWNvbmQgbGFuZ3VhZ2UgZGV2
ZWxvcG1lbnQgaW4gd3JpdGluZzogTWVhc3VyZXMgb2YgZmx1ZW5jeSwgYWNjdXJhY3kgJmFtcDsg
Y29tcGxleGl0eTwvdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE4NzwvcGFnZXM+PGRhdGVzPjx5ZWFy
PjE5OTg8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkhvbm9sdWx1LCBISTwvcHViLWxvY2F0
aW9uPjxwdWJsaXNoZXI+VW5pdmVyc2l0eSBvZiBIYXdhaSZhcG9zO2ksIFNlY29uZCBMYW5ndWFn
ZSBUZWFjaGluZyBhbmQgQ3VycmljdWx1bSBDZW50ZXI8L3B1Ymxpc2hlcj48aXNibj4xNzwvaXNi
bj48Y2FsbC1udW0+UDUzIFc2NSAxOTk4PC9jYWxsLW51bT48d29yay10eXBlPlRlY2huaWNhbCBS
ZXBvcnQ8L3dvcmstdHlwZT48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT4A
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Jd2FzaGl0YTwvQXV0aG9yPjxZZWFyPjIwMDg8L1llYXI+
PFJlY051bT4yMTY8L1JlY051bT48RGlzcGxheVRleHQ+KGUuZy4sIEN1bW1pbmcgZXQgYWwuLCAy
MDA1OyBFbGxpcyAmYW1wOyBZdWFuLCAyMDA0OyBIb3VzZW4gJmFtcDsgS3Vpa2VuLCAyMDA5Yjsg
SXdhc2hpdGEsIEJyb3duLCBNY05hbWFyYSwgJmFtcDsgTyZhcG9zO0hhZ2FuLCAyMDA4OyBLdWlr
ZW4gJmFtcDsgVmVkZGVyLCAyMDA4OyBMaSwgMjAwMDsgU3RvcmNoLCAyMDA5OyBXaWdnbGVzd29y
dGggJmFtcDsgU3RvcmNoLCAyMDA5OyBXb2xmZS1RdWludGVybywgSW5hZ2FraSwgJmFtcDsgS2lt
LCAxOTk4KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4yMTY8L3JlYy1udW1iZXI+
PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5
cnVzcGF4eHhyZnpmc3ciPjIxNjwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0
cWdjQUFBcHZNQ0UiPjcxOTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3Vy
bmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+
SXdhc2hpdGEsIE5vcmlrbzwvYXV0aG9yPjxhdXRob3I+QnJvd24sIEFubmllPC9hdXRob3I+PGF1
dGhvcj5NY05hbWFyYSwgVGltPC9hdXRob3I+PGF1dGhvcj5PJmFwb3M7SGFnYW4sIFNhbGx5PC9h
dXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc2Vzc2VkIGxl
dmVscyBvZiBzZWNvbmQgbGFuZ3VhZ2Ugc3BlYWtpbmcgcHJvZmljaWVuY3k6IEhvdyBkaXN0aW5j
dD88L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5
LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3Rp
Y3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz4yNC00OTwvcGFnZXM+PHZvbHVtZT4y
OTwvdm9sdW1lPjxudW1iZXI+MTwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDA4PC95ZWFyPjxwdWIt
ZGF0ZXM+PGRhdGU+TWFyY2ggMSwgMjAwODwvZGF0ZT48L3B1Yi1kYXRlcz48L2RhdGVzPjx1cmxz
PjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vYXBwbGlqLm94Zm9yZGpvdXJuYWxzLm9yZy9jZ2kv
Y29udGVudC9hYnN0cmFjdC8yOS8xLzI0IDwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxl
Y3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA5My9hcHBsaW4vYW1tMDE3PC9lbGVjdHJvbmljLXJl
c291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Ib3VzZW48L0F1dGhvcj48
WWVhcj4yMDA5PC9ZZWFyPjxSZWNOdW0+NDEzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40
MTM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5
NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjQxMzwva2V5PjwvZm9yZWlnbi1rZXlzPjxy
ZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3Jz
PjxhdXRob3JzPjxhdXRob3I+SG91c2VuLCBBbGV4PC9hdXRob3I+PGF1dGhvcj5LdWlrZW4sIEZv
bGtlcnQ8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+Q29t
cGxleGl0eSwgYWNjdXJhY3ksIGFuZCBmbHVlbmN5IGluIHNlY29uZCBsYW5ndWFnZSBhY3F1aXNp
dGlvbiBbU3BlY2lhbCBpc3N1ZV08L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBwbGllZCBMaW5n
dWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxl
PkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz40NjEg
LSA2MjYgPC9wYWdlcz48dm9sdW1lPjMwPC92b2x1bWU+PG51bWJlcj40PC9udW1iZXI+PGRhdGVz
Pjx5ZWFyPjIwMDk8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PGN1c3RvbTI+QXBwbGllZCBM
aW5ndWlzdGljczwvY3VzdG9tMj48Y3VzdG9tMz5Ib3VzZW4sIEEuLCAmYW1wOyBLdWlrZW4sIEYu
IChFZHMuKTwvY3VzdG9tMz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5DdW1taW5nPC9B
dXRob3I+PFllYXI+MjAwNTwvWWVhcj48UmVjTnVtPjI2NzwvUmVjTnVtPjxQcmVmaXg+ZS5nLmAs
IDwvUHJlZml4PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjY3PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl
eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6
ZnN3Ij4yNjc8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNF
Ij43Njk8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xl
Ij4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkN1bW1pbmcsIEFs
aXN0ZXI8L2F1dGhvcj48YXV0aG9yPkthbnRvciwgUm9iZXJ0PC9hdXRob3I+PGF1dGhvcj5CYWJh
LCBLeW9rbzwvYXV0aG9yPjxhdXRob3I+RXJkb3N5LCBVc21hbjwvYXV0aG9yPjxhdXRob3I+RW91
YW56b3VpLCBLZWFucmU8L2F1dGhvcj48YXV0aG9yPkphbWVzLCBNYXJrPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkRpZmZlcmVuY2VzIGluIHdyaXR0ZW4g
ZGlzY291cnNlIGluIGluZGVwZW5kZW50IGFuZCBpbnRlZ3JhdGVkIHByb3RvdHlwZSB0YXNrcyBm
b3IgbmV4dCBnZW5lcmF0aW9uIFRPRUZMPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFzc2Vzc2lu
ZyBXcml0aW5nPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0
bGU+QXNzZXNzaW5nIFdyaXRpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz41LTQz
PC9wYWdlcz48dm9sdW1lPjEwPC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGtleXdvcmRzPjxr
ZXl3b3JkPldyaXRpbmcgYXNzZXNzbWVudDwva2V5d29yZD48a2V5d29yZD5UT0VGTDwva2V5d29y
ZD48a2V5d29yZD5JbnRlZ3JhdGVkIHRhc2tzPC9rZXl3b3JkPjxrZXl3b3JkPkRpc2NvdXJzZSBh
bmFseXNpczwva2V5d29yZD48a2V5d29yZD5FU0wgd3JpdGluZzwva2V5d29yZD48L2tleXdvcmRz
PjxkYXRlcz48eWVhcj4yMDA1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTA3NS0yOTM1PC9pc2JuPjx1
cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vd3d3LnNjaWVuY2VkaXJlY3QuY29tL3NjaWVu
Y2UvYXJ0aWNsZS9CNlZUOC00RlhOUlBDLTEvMi8wNGE3MTk1NzRhMGY2OGFiZGUxMTQzZmZkYjRk
ZWRiNzwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+
MTAuMTAxNi9qLmFzdy4yMDA1LjAyLjAwMTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNv
cmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+RWxsaXM8L0F1dGhvcj48WWVhcj4yMDA0PC9ZZWFyPjxS
ZWNOdW0+NjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjYxPC9yZWMtbnVtYmVyPjxmb3Jl
aWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3Bh
eHh4cmZ6ZnN3Ij42MTwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFB
cHZNQ0UiPjU3Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFy
dGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+RWxsaXMs
IFJvZDwvYXV0aG9yPjxhdXRob3I+WXVhbiwgRmFuZ3l1YW48L2F1dGhvcj48L2F1dGhvcnM+PC9j
b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VGhlIGVmZmVjdHMgb2YgcGxhbm5pbmcgb24gZmx1
ZW5jeSwgY29tcGxleGl0eSwgYW5kIGFjY3VyYWN5IGluIHNlY29uZCBsYW5ndWFnZSBuYXJyYXRp
dmUgd3JpdGluZzwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TdHVkaWVzIGluIFNlY29uZCBMYW5n
dWFnZSBBY3F1aXNpdGlvbjwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxm
dWxsLXRpdGxlPlN0dWRpZXMgaW4gU2Vjb25kIExhbmd1YWdlIEFjcXVpc2l0aW9uPC9mdWxsLXRp
dGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NTktODQ8L3BhZ2VzPjx2b2x1bWU+MjY8L3ZvbHVtZT48
bnVtYmVyPjE8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwNDwveWVhcj48L2RhdGVzPjx1cmxzPjwv
dXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTAxNy9TMDI3MjI2MzEwNDAyNjEzMDwv
ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+S3Vp
a2VuPC9BdXRob3I+PFllYXI+MjAwODwvWWVhcj48UmVjTnVtPjIxMTwvUmVjTnVtPjxyZWNvcmQ+
PHJlYy1udW1iZXI+MjExPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yMTE8L2tleT48a2V5
IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43MTQ8L2tleT48L2ZvcmVp
Z24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNv
bnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkt1aWtlbiwgRm9sa2VydDwvYXV0aG9yPjxhdXRo
b3I+VmVkZGVyLCBJbmVrZTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5Db2duaXRpdmUgdGFzayBjb21wbGV4aXR5IGFuZCB3cml0dGVuIG91dHB1dCBpbiBJ
dGFsaWFuIGFuZCBGcmVuY2ggYXMgYSBmb3JlaWduIGxhbmd1YWdlPC90aXRsZT48c2Vjb25kYXJ5
LXRpdGxlPkpvdXJuYWwgb2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNlY29uZCBMYW5n
dWFnZSBXcml0aW5nPC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NDgtNjA8L3BhZ2Vz
Pjx2b2x1bWU+MTc8L3ZvbHVtZT48bnVtYmVyPjE8L251bWJlcj48a2V5d29yZHM+PGtleXdvcmQ+
QWNjdXJhY3k8L2tleXdvcmQ+PGtleXdvcmQ+QXR0ZW50aW9uPC9rZXl3b3JkPjxrZXl3b3JkPkNv
Z25pdGlvbiBIeXBvdGhlc2lzPC9rZXl3b3JkPjxrZXl3b3JkPkNvZ25pdGl2ZSB0YXNrIGNvbXBs
ZXhpdHk8L2tleXdvcmQ+PGtleXdvcmQ+TGV4aWNhbCB2YXJpYXRpb248L2tleXdvcmQ+PGtleXdv
cmQ+TGltaXRlZCBBdHRlbnRpb25hbCBDYXBhY2l0eSBNb2RlbDwva2V5d29yZD48a2V5d29yZD5N
dWx0aXBsZSBBdHRlbnRpb25hbCBSZXNvdXJjZXMgTW9kZWw8L2tleXdvcmQ+PGtleXdvcmQ+UHJv
ZmljaWVuY3kgbGV2ZWw8L2tleXdvcmQ+PGtleXdvcmQ+U3ludGFjdGljIGNvbXBsZXhpdHk8L2tl
eXdvcmQ+PGtleXdvcmQ+V3JpdGluZyBwZXJmb3JtYW5jZTwva2V5d29yZD48L2tleXdvcmRzPjxk
YXRlcz48eWVhcj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0
dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20vc2NpZW5jZS9hcnRpY2xlL0I2VzVGLTRSMTFLRkMt
My8xL2NlNzNlNGI0NWQ0MjY5NzY0OTc4MjE4MWUwMTRiYTNhIDwvdXJsPjwvcmVsYXRlZC11cmxz
PjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTAxNi9qLmpzbHcuMjAwNy4wOC4w
MDMgPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhv
cj5MaTwvQXV0aG9yPjxZZWFyPjIwMDA8L1llYXI+PFJlY051bT4xNjI8L1JlY051bT48cmVjb3Jk
PjxyZWMtbnVtYmVyPjE2MjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIg
ZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+MTYyPC9rZXk+PGtl
eSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NjY5PC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5MaSwgWWlsaTwvYXV0aG9yPjwvYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5MaW5ndWlzdGljIGNoYXJhY3RlcmlzdGljcyBv
ZiBFU0wgd3JpdGluZyBpbiB0YXNrLWJhc2VkIGUtbWFpbCBhY3Rpdml0aWVzPC90aXRsZT48c2Vj
b25kYXJ5LXRpdGxlPlN5c3RlbTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2Fs
PjxmdWxsLXRpdGxlPlN5c3RlbTwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjIyOS0y
NDU8L3BhZ2VzPjx2b2x1bWU+Mjg8L3ZvbHVtZT48bnVtYmVyPjI8L251bWJlcj48ZGF0ZXM+PHll
YXI+MjAwMDwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1u
dW0+MTAuMTAxNi9TMDM0Ni0yNTFYKDAwKTAwMDA5LTkgPC9lbGVjdHJvbmljLXJlc291cmNlLW51
bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5TdG9yY2g8L0F1dGhvcj48WWVhcj4yMDA5
PC9ZZWFyPjxSZWNOdW0+MzgyPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zODI8L3JlYy1u
dW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2Vh
djBwcDk5cnVzcGF4eHhyZnpmc3ciPjM4Mjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu
YW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz
PjxhdXRob3I+U3RvcmNoLCBOZW9teTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48
dGl0bGVzPjx0aXRsZT5UaGUgaW1wYWN0IG9mIHN0dWR5aW5nIGluIGEgc2Vjb25kIGxhbmd1YWdl
IChMMikgbWVkaXVtIHVuaXZlcnNpdHkgb24gdGhlIGRldmVsb3BtZW50IG9mIEwyIHdyaXRpbmc8
L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBTZWNvbmQgTGFuZ3VhZ2UgV3JpdGlu
Zzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkpvdXJu
YWwgb2YgU2Vjb25kIExhbmd1YWdlIFdyaXRpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxw
YWdlcz4xMDMtMTE4PC9wYWdlcz48dm9sdW1lPjE4PC92b2x1bWU+PG51bWJlcj4yPC9udW1iZXI+
PGtleXdvcmRzPjxrZXl3b3JkPlNlY29uZCBsYW5ndWFnZSB3cml0aW5nIGRldmVsb3BtZW50PC9r
ZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PC9kYXRlcz48aXNibj4x
MDYwLTM3NDM8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5j
ZWRpcmVjdC5jb20vc2NpZW5jZS9hcnRpY2xlL0I2VzVGLTRWWENHNUstMS8yLzI5NWUxYWM1MzU0
NDI5MjdjZDA5ODk5NjM1OTkyNzM1PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJv
bmljLXJlc291cmNlLW51bT4xMC4xMDE2L2ouanNsdy4yMDA5LjAyLjAwMzwvZWxlY3Ryb25pYy1y
ZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V2lnZ2xlc3dvcnRoPC9B
dXRob3I+PFllYXI+MjAwOTwvWWVhcj48UmVjTnVtPjI2NjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1u
dW1iZXI+MjY2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
ZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yNjY8L2tleT48a2V5IGFwcD0i
RU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43Njg8L2tleT48L2ZvcmVpZ24ta2V5
cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1
dG9ycz48YXV0aG9ycz48YXV0aG9yPldpZ2dsZXN3b3J0aCwgR2lsbGlhbjwvYXV0aG9yPjxhdXRo
b3I+U3RvcmNoLCBOZW9teTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5QYWlyIHZlcnN1cyBpbmRpdmlkdWFsIHdyaXRpbmc6IEVmZmVjdHMgb24gZmx1ZW5j
eSwgY29tcGxleGl0eSBhbmQgYWNjdXJhY3k8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+TGFuZ3Vh
Z2UgVGVzdGluZzwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRp
dGxlPkxhbmd1YWdlIFRlc3Rpbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz40NDUt
NDY2PC9wYWdlcz48dm9sdW1lPjI2PC92b2x1bWU+PG51bWJlcj4zPC9udW1iZXI+PGRhdGVzPjx5
ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5KdWx5IDEsIDIwMDk8L2RhdGU+PC9wdWIt
ZGF0ZXM+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2x0ai5zYWdlcHVi
LmNvbS9jZ2kvY29udGVudC9hYnN0cmFjdC8yNi8zLzQ0NTwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTE3Ny8wMjY1NTMyMjA5MTA0NjcwPC9l
bGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Xb2xm
ZS1RdWludGVybzwvQXV0aG9yPjxZZWFyPjE5OTg8L1llYXI+PFJlY051bT4xMzwvUmVjTnVtPjxy
ZWNvcmQ+PHJlYy1udW1iZXI+MTM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i
RU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjEzPC9rZXk+
PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM4PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRv
cnM+PGF1dGhvcnM+PGF1dGhvcj5Xb2xmZS1RdWludGVybywgS2F0ZTwvYXV0aG9yPjxhdXRob3I+
SW5hZ2FraSwgU2h1bmppPC9hdXRob3I+PGF1dGhvcj5LaW0sIEhhZS1Zb3VuZzwvYXV0aG9yPjwv
YXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5TZWNvbmQgbGFuZ3VhZ2UgZGV2
ZWxvcG1lbnQgaW4gd3JpdGluZzogTWVhc3VyZXMgb2YgZmx1ZW5jeSwgYWNjdXJhY3kgJmFtcDsg
Y29tcGxleGl0eTwvdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE4NzwvcGFnZXM+PGRhdGVzPjx5ZWFy
PjE5OTg8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkhvbm9sdWx1LCBISTwvcHViLWxvY2F0
aW9uPjxwdWJsaXNoZXI+VW5pdmVyc2l0eSBvZiBIYXdhaSZhcG9zO2ksIFNlY29uZCBMYW5ndWFn
ZSBUZWFjaGluZyBhbmQgQ3VycmljdWx1bSBDZW50ZXI8L3B1Ymxpc2hlcj48aXNibj4xNzwvaXNi
bj48Y2FsbC1udW0+UDUzIFc2NSAxOTk4PC9jYWxsLW51bT48d29yay10eXBlPlRlY2huaWNhbCBS
ZXBvcnQ8L3dvcmstdHlwZT48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT4A
ADDIN EN.CITE.DATA (e.g., Cumming et al., 2005; Ellis & Yuan, 2004; Housen & Kuiken, 2009b; Iwashita, Brown, McNamara, & O'Hagan, 2008; Kuiken & Vedder, 2008; Li, 2000; Storch, 2009; Wigglesworth & Storch, 2009; Wolfe-Quintero, Inagaki, & Kim, 1998). The expectation is that higher levels of accuracy and complexity are associated with increased proficiency or higher performance levels on a rating scale PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Qb2xpbzwvQXV0aG9yPjxZZWFyPjE5OTc8L1llYXI+PFJl
Y051bT42ODwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oZS5nLiwgRWR1Y2F0aW9uYWwgVGVzdGluZyBT
ZXJ2aWNlLCAyMDA0OyBJbnRlcm5hdGlvbmFsIEVuZ2xpc2ggTGFuZ3VhZ2UgVGVzdGluZyBTeXN0
ZW0sIG4uZC4tYSwgbi5kLi1iOyBQb2xpbywgMTk5NzsgV29sZmUtUXVpbnRlcm8gZXQgYWwuLCAx
OTk4KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj42ODwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNw
YXh4eHJmemZzdyI+Njg8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FB
QXB2TUNFIj41ODM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBB
cnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlBvbGlv
LCBDaGFybGVuZSBHLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0
aXRsZT5NZWFzdXJlcyBvZiBsaW5ndWlzdGljIGFjY3VyYWN5IGluIHNlY29uZCBsYW5ndWFnZSB3
cml0aW5nIHJlc2VhcmNoPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkxhbmd1YWdlIExlYXJuaW5n
PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TGFuZ3Vh
Z2UgTGVhcm5pbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMDEtMTQzPC9wYWdl
cz48dm9sdW1lPjQ3PC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjE5OTc8
L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEx
MTEvMDAyMy04MzMzLjMxOTk3MDAzPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48
L0NpdGU+PENpdGU+PEF1dGhvcj5FZHVjYXRpb25hbCBUZXN0aW5nIFNlcnZpY2U8L0F1dGhvcj48
WWVhcj4yMDA0PC9ZZWFyPjxSZWNOdW0+MTc4PC9SZWNOdW0+PFByZWZpeD5lLmcuYCwgPC9QcmVm
aXg+PHJlY29yZD48cmVjLW51bWJlcj4xNzg8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjE3
ODwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjY4NTwv
a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJXZWIgUGFnZSI+MTI8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5FZHVjYXRpb25hbCBUZXN0aW5nIFNlcnZp
Y2UsPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlNjb3Jp
bmcgR3VpZGVzIChSdWJyaWNzKSBmb3IgV3JpdGluZyBSZXNwb25zZXMuPC90aXRsZT48L3RpdGxl
cz48bnVtYmVyPkFwcmlsIDgsIDIwMDc8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwNDwveWVhcj48
L2RhdGVzPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD53d3cuZXRzLm9yZyB1bmRlciAmcXVvdDtp
QlQgVE9FRkwgU2NvcmluZyBHdWlkZXMmcXVvdDs8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+
PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+SW50ZXJuYXRpb25hbCBFbmdsaXNoIExhbmd1
YWdlIFRlc3RpbmcgU3lzdGVtPC9BdXRob3I+PFllYXI+bi5kLjwvWWVhcj48UmVjTnVtPjI1MTwv
UmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjUxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+
PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3
Ij4yNTE8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43
NTM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iV2ViIFBhZ2UiPjEyPC9yZWYt
dHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+SW50ZXJuYXRpb25hbCBFbmdsaXNo
IExhbmd1YWdlIFRlc3RpbmcgU3lzdGVtLDwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9y
cz48dGl0bGVzPjx0aXRsZT5JRUxUUyBUYXNrIDIgd3JpdGluZyBiYW5kIGRlc2NyaXB0b3JzLjwv
dGl0bGU+PC90aXRsZXM+PG51bWJlcj5KdW5lIDEwLCAyMDA5PC9udW1iZXI+PGRhdGVzPjx5ZWFy
Pm4uZC48L3llYXI+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL3d3dy5p
ZWx0cy5vcmcvcmVzZWFyY2hlcnMvc2NvcmVfcHJvY2Vzc2luZ19hbmRfcmVwb3J0aW5nLmFzcHg8
L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+
SW50ZXJuYXRpb25hbCBFbmdsaXNoIExhbmd1YWdlIFRlc3RpbmcgU3lzdGVtPC9BdXRob3I+PFll
YXI+bi5kLjwvWWVhcj48UmVjTnVtPjI1MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjUy
PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2
d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yNTI8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRi
LWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43NTQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5
cGUgbmFtZT0iV2ViIFBhZ2UiPjEyPC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+SW50ZXJuYXRpb25hbCBFbmdsaXNoIExhbmd1YWdlIFRlc3RpbmcgU3lzdGVtLDwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JRUxUUyBUYXNrIDEg
d3JpdGluZyBiYW5kIGRlc2NyaXB0b3JzLjwvdGl0bGU+PC90aXRsZXM+PG51bWJlcj5KdW5lIDEw
LCAyMDA5PC9udW1iZXI+PGRhdGVzPjx5ZWFyPm4uZC48L3llYXI+PC9kYXRlcz48dXJscz48cmVs
YXRlZC11cmxzPjx1cmw+aHR0cDovL3d3dy5pZWx0cy5vcmcvcmVzZWFyY2hlcnMvc2NvcmVfcHJv
Y2Vzc2luZ19hbmRfcmVwb3J0aW5nLmFzcHg8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V29sZmUtUXVpbnRlcm88L0F1dGhvcj48WWVhcj4x
OTk4PC9ZZWFyPjxSZWNOdW0+MTM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjEzPC9yZWMt
bnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdl
YXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4xMzwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNn
UkRld3J0cWdjQUFBcHZNQ0UiPjUzODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1l
PSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+V29sZmUt
UXVpbnRlcm8sIEthdGU8L2F1dGhvcj48YXV0aG9yPkluYWdha2ksIFNodW5qaTwvYXV0aG9yPjxh
dXRob3I+S2ltLCBIYWUtWW91bmc8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRp
dGxlcz48dGl0bGU+U2Vjb25kIGxhbmd1YWdlIGRldmVsb3BtZW50IGluIHdyaXRpbmc6IE1lYXN1
cmVzIG9mIGZsdWVuY3ksIGFjY3VyYWN5ICZhbXA7IGNvbXBsZXhpdHk8L3RpdGxlPjwvdGl0bGVz
PjxwYWdlcz4xODc8L3BhZ2VzPjxkYXRlcz48eWVhcj4xOTk4PC95ZWFyPjwvZGF0ZXM+PHB1Yi1s
b2NhdGlvbj5Ib25vbHVsdSwgSEk8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlVuaXZlcnNpdHkg
b2YgSGF3YWkmYXBvcztpLCBTZWNvbmQgTGFuZ3VhZ2UgVGVhY2hpbmcgYW5kIEN1cnJpY3VsdW0g
Q2VudGVyPC9wdWJsaXNoZXI+PGlzYm4+MTc8L2lzYm4+PGNhbGwtbnVtPlA1MyBXNjUgMTk5ODwv
Y2FsbC1udW0+PHdvcmstdHlwZT5UZWNobmljYWwgUmVwb3J0PC93b3JrLXR5cGU+PHVybHM+PC91
cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+AG==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Qb2xpbzwvQXV0aG9yPjxZZWFyPjE5OTc8L1llYXI+PFJl
Y051bT42ODwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oZS5nLiwgRWR1Y2F0aW9uYWwgVGVzdGluZyBT
ZXJ2aWNlLCAyMDA0OyBJbnRlcm5hdGlvbmFsIEVuZ2xpc2ggTGFuZ3VhZ2UgVGVzdGluZyBTeXN0
ZW0sIG4uZC4tYSwgbi5kLi1iOyBQb2xpbywgMTk5NzsgV29sZmUtUXVpbnRlcm8gZXQgYWwuLCAx
OTk4KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj42ODwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNw
YXh4eHJmemZzdyI+Njg8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FB
QXB2TUNFIj41ODM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBB
cnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlBvbGlv
LCBDaGFybGVuZSBHLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0
aXRsZT5NZWFzdXJlcyBvZiBsaW5ndWlzdGljIGFjY3VyYWN5IGluIHNlY29uZCBsYW5ndWFnZSB3
cml0aW5nIHJlc2VhcmNoPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkxhbmd1YWdlIExlYXJuaW5n
PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TGFuZ3Vh
Z2UgTGVhcm5pbmc8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMDEtMTQzPC9wYWdl
cz48dm9sdW1lPjQ3PC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjE5OTc8
L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEx
MTEvMDAyMy04MzMzLjMxOTk3MDAzPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48
L0NpdGU+PENpdGU+PEF1dGhvcj5FZHVjYXRpb25hbCBUZXN0aW5nIFNlcnZpY2U8L0F1dGhvcj48
WWVhcj4yMDA0PC9ZZWFyPjxSZWNOdW0+MTc4PC9SZWNOdW0+PFByZWZpeD5lLmcuYCwgPC9QcmVm
aXg+PHJlY29yZD48cmVjLW51bWJlcj4xNzg8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjE3
ODwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjY4NTwv
a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJXZWIgUGFnZSI+MTI8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5FZHVjYXRpb25hbCBUZXN0aW5nIFNlcnZp
Y2UsPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlNjb3Jp
bmcgR3VpZGVzIChSdWJyaWNzKSBmb3IgV3JpdGluZyBSZXNwb25zZXMuPC90aXRsZT48L3RpdGxl
cz48bnVtYmVyPkFwcmlsIDgsIDIwMDc8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwNDwveWVhcj48
L2RhdGVzPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD53d3cuZXRzLm9yZyB1bmRlciAmcXVvdDtp
QlQgVE9FRkwgU2NvcmluZyBHdWlkZXMmcXVvdDs8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+
PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+SW50ZXJuYXRpb25hbCBFbmdsaXNoIExhbmd1
YWdlIFRlc3RpbmcgU3lzdGVtPC9BdXRob3I+PFllYXI+bi5kLjwvWWVhcj48UmVjTnVtPjI1MTwv
UmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjUxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+
PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3
Ij4yNTE8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43
NTM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iV2ViIFBhZ2UiPjEyPC9yZWYt
dHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+SW50ZXJuYXRpb25hbCBFbmdsaXNo
IExhbmd1YWdlIFRlc3RpbmcgU3lzdGVtLDwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9y
cz48dGl0bGVzPjx0aXRsZT5JRUxUUyBUYXNrIDIgd3JpdGluZyBiYW5kIGRlc2NyaXB0b3JzLjwv
dGl0bGU+PC90aXRsZXM+PG51bWJlcj5KdW5lIDEwLCAyMDA5PC9udW1iZXI+PGRhdGVzPjx5ZWFy
Pm4uZC48L3llYXI+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL3d3dy5p
ZWx0cy5vcmcvcmVzZWFyY2hlcnMvc2NvcmVfcHJvY2Vzc2luZ19hbmRfcmVwb3J0aW5nLmFzcHg8
L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+
SW50ZXJuYXRpb25hbCBFbmdsaXNoIExhbmd1YWdlIFRlc3RpbmcgU3lzdGVtPC9BdXRob3I+PFll
YXI+bi5kLjwvWWVhcj48UmVjTnVtPjI1MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjUy
PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2
d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yNTI8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRi
LWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43NTQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5
cGUgbmFtZT0iV2ViIFBhZ2UiPjEyPC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+SW50ZXJuYXRpb25hbCBFbmdsaXNoIExhbmd1YWdlIFRlc3RpbmcgU3lzdGVtLDwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JRUxUUyBUYXNrIDEg
d3JpdGluZyBiYW5kIGRlc2NyaXB0b3JzLjwvdGl0bGU+PC90aXRsZXM+PG51bWJlcj5KdW5lIDEw
LCAyMDA5PC9udW1iZXI+PGRhdGVzPjx5ZWFyPm4uZC48L3llYXI+PC9kYXRlcz48dXJscz48cmVs
YXRlZC11cmxzPjx1cmw+aHR0cDovL3d3dy5pZWx0cy5vcmcvcmVzZWFyY2hlcnMvc2NvcmVfcHJv
Y2Vzc2luZ19hbmRfcmVwb3J0aW5nLmFzcHg8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V29sZmUtUXVpbnRlcm88L0F1dGhvcj48WWVhcj4x
OTk4PC9ZZWFyPjxSZWNOdW0+MTM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjEzPC9yZWMt
bnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdl
YXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4xMzwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNn
UkRld3J0cWdjQUFBcHZNQ0UiPjUzODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1l
PSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+V29sZmUt
UXVpbnRlcm8sIEthdGU8L2F1dGhvcj48YXV0aG9yPkluYWdha2ksIFNodW5qaTwvYXV0aG9yPjxh
dXRob3I+S2ltLCBIYWUtWW91bmc8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRp
dGxlcz48dGl0bGU+U2Vjb25kIGxhbmd1YWdlIGRldmVsb3BtZW50IGluIHdyaXRpbmc6IE1lYXN1
cmVzIG9mIGZsdWVuY3ksIGFjY3VyYWN5ICZhbXA7IGNvbXBsZXhpdHk8L3RpdGxlPjwvdGl0bGVz
PjxwYWdlcz4xODc8L3BhZ2VzPjxkYXRlcz48eWVhcj4xOTk4PC95ZWFyPjwvZGF0ZXM+PHB1Yi1s
b2NhdGlvbj5Ib25vbHVsdSwgSEk8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlVuaXZlcnNpdHkg
b2YgSGF3YWkmYXBvcztpLCBTZWNvbmQgTGFuZ3VhZ2UgVGVhY2hpbmcgYW5kIEN1cnJpY3VsdW0g
Q2VudGVyPC9wdWJsaXNoZXI+PGlzYm4+MTc8L2lzYm4+PGNhbGwtbnVtPlA1MyBXNjUgMTk5ODwv
Y2FsbC1udW0+PHdvcmstdHlwZT5UZWNobmljYWwgUmVwb3J0PC93b3JrLXR5cGU+PHVybHM+PC91
cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+AG==
ADDIN EN.CITE.DATA (e.g., Educational Testing Service, 2004; International English Language Testing System, n.d.-a, n.d.-b; Polio, 1997; Wolfe-Quintero et al., 1998).Grammatical complexity is an important complementary construct to grammatical accuracy in order to operationalize grammatical ability comprehensively as both control (accuracy) and range (complexity) of the linguistic forms referred to in the language ability models mentioned above need to be assessed. In addition, the analysis of grammatical and/or syntactic complexity is important “because of the assumption that language development entails, among other processes, the growth of an L2 learner’s syntactic repertoire and her or his ability to use that repertoire appropriately in a variety of situations” ADDIN EN.CITE <EndNote><Cite><Author>Ortega</Author><Year>2003</Year><RecNum>66</RecNum><Pages>492</Pages><DisplayText>(Ortega, 2003, p. 492)</DisplayText><record><rec-number>66</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">66</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">581</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Ortega, Lourdes</author></authors></contributors><titles><title>Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>492-518</pages><volume>24</volume><number>4</number><dates><year>2003</year></dates><urls><related-urls><url> </electronic-resource-num></record></Cite></EndNote>(Ortega, 2003, p. 492). This means (a) that learners have both basic and sophisticated structures at their disposal as their grammatical ability and proficiency increase and (b) that (it is assumed) L2 writers can then choose the structure that best fits the context and the purpose of the communicative situation ADDIN EN.CITE <EndNote><Cite><Author>Wolfe-Quintero</Author><Year>1998</Year><RecNum>13</RecNum><DisplayText>(Wolfe-Quintero et al., 1998)</DisplayText><record><rec-number>13</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">13</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">538</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Wolfe-Quintero, Kate</author><author>Inagaki, Shunji</author><author>Kim, Hae-Young</author></authors></contributors><titles><title>Second language development in writing: Measures of fluency, accuracy & complexity</title></titles><pages>187</pages><dates><year>1998</year></dates><pub-location>Honolulu, HI</pub-location><publisher>University of Hawai'i, Second Language Teaching and Curriculum Center</publisher><isbn>17</isbn><call-num>P53 W65 1998</call-num><work-type>Technical Report</work-type><urls></urls></record></Cite></EndNote>(Wolfe-Quintero et al., 1998). However, both Beers and Nagy ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Beers</Author><Year>2009</Year><RecNum>244</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>244</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">244</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">747</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Beers, Scott</author><author>Nagy, William</author></authors></contributors><titles><title>Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre?</title><secondary-title>Reading and Writing</secondary-title></titles><pages>185-200</pages><volume>22</volume><number>2</number><dates><year>2009</year></dates><urls><related-urls><url>;(2009) and Ortega ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Ortega</Author><Year>2003</Year><RecNum>66</RecNum><DisplayText>(2003)</DisplayText><record><rec-number>66</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">66</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">581</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Ortega, Lourdes</author></authors></contributors><titles><title>Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>492-518</pages><volume>24</volume><number>4</number><dates><year>2003</year></dates><urls><related-urls><url> </electronic-resource-num></record></Cite></EndNote>(2003) caution not to assume that linguistically more complex language leads to higher quality texts and argue complexity should be considered a necessary but not sufficient condition for good writing.Another reason to assess both accuracy and complexity is the trade-off between accuracy and complexity for oral, task-based contexts in language performance PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Ta2VoYW48L0F1dGhvcj48WWVhcj4yMDA5PC9ZZWFyPjxS
ZWNOdW0+MzYzPC9SZWNOdW0+PERpc3BsYXlUZXh0PihTa2VoYW4sIDE5OTYsIDE5OTgsIDIwMDkp
PC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjM2MzwvcmVjLW51bWJlcj48Zm9yZWln
bi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4
eHJmemZzdyI+MzYzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwg
QXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Ta2Vo
YW4sIFBldGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxl
Pk1vZGVsbGluZyBzZWNvbmQgbGFuZ3VhZ2UgcGVyZm9ybWFuY2U6IEludGVncmF0aW5nIGNvbXBs
ZXhpdHksIGFjY3VyYWN5LCBmbHVlbmN5LCBhbmQgbGV4aXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0
bGU+QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2Rp
Y2FsPjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2Rp
Y2FsPjxwYWdlcz41MTAtNTMyPC9wYWdlcz48dm9sdW1lPjMwPC92b2x1bWU+PG51bWJlcj40PC9u
dW1iZXI+PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5EZWNlbWJlciAx
LCAyMDA5PC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJs
Pmh0dHA6Ly9hcHBsaWoub3hmb3Jkam91cm5hbHMub3JnL2NnaS9jb250ZW50L2Fic3RyYWN0LzMw
LzQvNTEwPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51
bT4xMC4xMDkzL2FwcGxpbi9hbXAwNDc8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3Jk
PjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNrZWhhbjwvQXV0aG9yPjxZZWFyPjE5OTY8L1llYXI+PFJl
Y051bT4zNzM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM3MzwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNw
YXh4eHJmemZzdyI+MzczPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJu
YWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5T
a2VoYW4sIFBldGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp
dGxlPkEgRnJhbWV3b3JrIGZvciB0aGUgaW1wbGVtZW50YXRpb24gb2YgdGFzay1iYXNlZCBpbnN0
cnVjdGlvbjwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5BcHBsaWVkIExpbmd1aXN0aWNzPC9zZWNv
bmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QXBwbGllZCBMaW5n
dWlzdGljczwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjM4LTYyPC9wYWdlcz48dm9s
dW1lPjE3PC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjE5OTY8L3llYXI+
PHB1Yi1kYXRlcz48ZGF0ZT5NYXJjaCAxLCAxOTk2PC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+
PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly9hcHBsaWoub3hmb3Jkam91cm5hbHMub3Jn
L2NnaS9jb250ZW50L2Fic3RyYWN0LzE3LzEvMzg8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+
PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwOTMvYXBwbGluLzE3LjEuMzg8L2VsZWN0cm9u
aWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNrZWhhbjwvQXV0
aG9yPjxZZWFyPjE5OTg8L1llYXI+PFJlY051bT4xMjA8L1JlY051bT48cmVjb3JkPjxyZWMtbnVt
YmVyPjEyMDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4
cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+MTIwPC9rZXk+PGtleSBhcHA9IkVO
V2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NjMxPC9rZXk+PC9mb3JlaWduLWtleXM+
PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+
PGF1dGhvcj5Ta2VoYW4sIFBldGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPkEgY29nbml0aXZlIGFwcHJvYWNoIHRvIGxhbmd1YWdlIGxlYXJuaW5nPC90
aXRsZT48L3RpdGxlcz48cGFnZXM+MzI0PC9wYWdlcz48ZGF0ZXM+PHllYXI+MTk5ODwveWVhcj48
L2RhdGVzPjxwdWItbG9jYXRpb24+T3hmb3JkPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5PeGZv
cmQgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjxpc2JuPjAtMTktNDM3MjE3LTA8L2lzYm4+
PGNhbGwtbnVtPlAxMTguMiBTNTM3IDIwMDM8L2NhbGwtbnVtPjx1cmxzPjwvdXJscz48L3JlY29y
ZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Ta2VoYW48L0F1dGhvcj48WWVhcj4yMDA5PC9ZZWFyPjxS
ZWNOdW0+MzYzPC9SZWNOdW0+PERpc3BsYXlUZXh0PihTa2VoYW4sIDE5OTYsIDE5OTgsIDIwMDkp
PC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjM2MzwvcmVjLW51bWJlcj48Zm9yZWln
bi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4
eHJmemZzdyI+MzYzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwg
QXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Ta2Vo
YW4sIFBldGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxl
Pk1vZGVsbGluZyBzZWNvbmQgbGFuZ3VhZ2UgcGVyZm9ybWFuY2U6IEludGVncmF0aW5nIGNvbXBs
ZXhpdHksIGFjY3VyYWN5LCBmbHVlbmN5LCBhbmQgbGV4aXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0
bGU+QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2Rp
Y2FsPjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2Rp
Y2FsPjxwYWdlcz41MTAtNTMyPC9wYWdlcz48dm9sdW1lPjMwPC92b2x1bWU+PG51bWJlcj40PC9u
dW1iZXI+PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5EZWNlbWJlciAx
LCAyMDA5PC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJs
Pmh0dHA6Ly9hcHBsaWoub3hmb3Jkam91cm5hbHMub3JnL2NnaS9jb250ZW50L2Fic3RyYWN0LzMw
LzQvNTEwPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51
bT4xMC4xMDkzL2FwcGxpbi9hbXAwNDc8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3Jk
PjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNrZWhhbjwvQXV0aG9yPjxZZWFyPjE5OTY8L1llYXI+PFJl
Y051bT4zNzM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM3MzwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNw
YXh4eHJmemZzdyI+MzczPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJu
YWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5T
a2VoYW4sIFBldGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp
dGxlPkEgRnJhbWV3b3JrIGZvciB0aGUgaW1wbGVtZW50YXRpb24gb2YgdGFzay1iYXNlZCBpbnN0
cnVjdGlvbjwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5BcHBsaWVkIExpbmd1aXN0aWNzPC9zZWNv
bmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QXBwbGllZCBMaW5n
dWlzdGljczwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjM4LTYyPC9wYWdlcz48dm9s
dW1lPjE3PC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjE5OTY8L3llYXI+
PHB1Yi1kYXRlcz48ZGF0ZT5NYXJjaCAxLCAxOTk2PC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+
PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly9hcHBsaWoub3hmb3Jkam91cm5hbHMub3Jn
L2NnaS9jb250ZW50L2Fic3RyYWN0LzE3LzEvMzg8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+
PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwOTMvYXBwbGluLzE3LjEuMzg8L2VsZWN0cm9u
aWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNrZWhhbjwvQXV0
aG9yPjxZZWFyPjE5OTg8L1llYXI+PFJlY051bT4xMjA8L1JlY051bT48cmVjb3JkPjxyZWMtbnVt
YmVyPjEyMDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4
cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+MTIwPC9rZXk+PGtleSBhcHA9IkVO
V2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NjMxPC9rZXk+PC9mb3JlaWduLWtleXM+
PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+
PGF1dGhvcj5Ta2VoYW4sIFBldGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPkEgY29nbml0aXZlIGFwcHJvYWNoIHRvIGxhbmd1YWdlIGxlYXJuaW5nPC90
aXRsZT48L3RpdGxlcz48cGFnZXM+MzI0PC9wYWdlcz48ZGF0ZXM+PHllYXI+MTk5ODwveWVhcj48
L2RhdGVzPjxwdWItbG9jYXRpb24+T3hmb3JkPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5PeGZv
cmQgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjxpc2JuPjAtMTktNDM3MjE3LTA8L2lzYm4+
PGNhbGwtbnVtPlAxMTguMiBTNTM3IDIwMDM8L2NhbGwtbnVtPjx1cmxzPjwvdXJscz48L3JlY29y
ZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE.DATA (Skehan, 1996, 1998, 2009). If the trade-off hypothesis also holds true for written tasks—and Li ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Li</Author><Year>2000</Year><RecNum>162</RecNum><DisplayText>(2000)</DisplayText><record><rec-number>162</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">162</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">669</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Li, Yili</author></authors></contributors><titles><title>Linguistic characteristics of ESL writing in task-based e-mail activities</title><secondary-title>System</secondary-title></titles><periodical><full-title>System</full-title></periodical><pages>229-245</pages><volume>28</volume><number>2</number><dates><year>2000</year></dates><urls></urls><electronic-resource-num>10.1016/S0346-251X(00)00009-9 </electronic-resource-num></record></Cite></EndNote>(2000) presents partial evidence for this—accuracy measures on their own do not tell the whole story about grammatical ability in L2 texts. Accuracy has to be considered jointly with complexity. Furthermore, even when complexity is not explicitly referred to in rating scale descriptors, there is evidence that raters of L2 writing attend to grammatical complexity in conjunction with grammatical accuracy when assessing language use in L2 texts ADDIN EN.CITE <EndNote><Cite><Author>Lumley</Author><Year>2005</Year><RecNum>334</RecNum><DisplayText>(Lumley, 2005)</DisplayText><record><rec-number>334</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">334</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Lumley, Tom</author></authors></contributors><titles><title>Assessing second language writing: The rater's perspective</title><secondary-title>Language testing and evaluation, 1612-815X ; v. 3</secondary-title></titles><number>368 p. :</number><keywords><keyword>English language Study and teaching Foreign speakers.</keyword><keyword>English language Ability testing.</keyword><keyword>English language Examinations Evaluation.</keyword></keywords><dates><year>2005</year></dates><pub-location>Frankfurt</pub-location><publisher>Peter Lang</publisher><isbn>0820476552 (US)
3631533276</isbn><urls></urls></record></Cite></EndNote>(Lumley, 2005). A combination of accuracy and complexity measures is, therefore, the most suitable text-based method to examine the teachers’ construct of grammatical ability in L2 writing.While grammatical accuracy is relatively easy to define as the absence of deviations from a particular linguistic norm ADDIN EN.CITE <EndNote><Cite><Author>Housen</Author><Year>2009</Year><RecNum>364</RecNum><DisplayText>(Housen & Kuiken, 2009a; Wolfe-Quintero et al., 1998)</DisplayText><record><rec-number>364</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">364</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Housen, Alex</author><author>Kuiken, Folkert</author></authors></contributors><titles><title>Complexity, accuracy, and fluency in second language acquisition</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>461-473</pages><volume>30</volume><number>4</number><dates><year>2009</year><pub-dates><date>December 1, 2009</date></pub-dates></dates><urls><related-urls><url> app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">13</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">538</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Wolfe-Quintero, Kate</author><author>Inagaki, Shunji</author><author>Kim, Hae-Young</author></authors></contributors><titles><title>Second language development in writing: Measures of fluency, accuracy & complexity</title></titles><pages>187</pages><dates><year>1998</year></dates><pub-location>Honolulu, HI</pub-location><publisher>University of Hawai'i, Second Language Teaching and Curriculum Center</publisher><isbn>17</isbn><call-num>P53 W65 1998</call-num><work-type>Technical Report</work-type><urls></urls></record></Cite></EndNote>(Housen & Kuiken, 2009a; Wolfe-Quintero et al., 1998), the concept of linguistic or grammatical complexity is a little more controversial because it can be seen in terms of size, range, and variety of linguistic resources at a learner’s disposal ADDIN EN.CITE <EndNote><Cite><Author>Housen</Author><Year>2009</Year><RecNum>364</RecNum><DisplayText>(Housen & Kuiken, 2009a)</DisplayText><record><rec-number>364</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">364</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Housen, Alex</author><author>Kuiken, Folkert</author></authors></contributors><titles><title>Complexity, accuracy, and fluency in second language acquisition</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>461-473</pages><volume>30</volume><number>4</number><dates><year>2009</year><pub-dates><date>December 1, 2009</date></pub-dates></dates><urls><related-urls><url>;(Housen & Kuiken, 2009a). In terms of operationalizing grammatical complexity, Norris and Ortega ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Norris</Author><Year>2009</Year><RecNum>366</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>366</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">366</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Norris, John M.</author><author>Ortega, Lourdes</author></authors></contributors><titles><title>Towards an organic approach to investigating CAF in instructed SLA: The case of complexity</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>555-578</pages><volume>30</volume><number>4</number><dates><year>2009</year><pub-dates><date>December 1, 2009</date></pub-dates></dates><urls><related-urls><url>;(2009) argue for a multidimensional definition of complexity that includes at the syntactic level measures of subordination and length, especially at the subclausal level, which points towards phrase-level elaboration. In other words, at least two measures of complexity are required in conjunction with accuracy measures for a comprehensive operationalization of grammatical ability.Voices of the Test Takers in L2 Writing Assessment ResearchLeki ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Leki</Author><Year>2001</Year><RecNum>326</RecNum><DisplayText>(2001)</DisplayText><record><rec-number>326</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">326</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">825</key></foreign-keys><ref-type name="Book Section">5</ref-type><contributors><authors><author>Leki, Ilona</author></authors><secondary-authors><author>Silva, Tony</author><author>Matsuda, Paul Kei</author></secondary-authors></contributors><titles><title>Hearing voices: L2 students' experiences in L2 writing courses</title><secondary-title>On second language writing</secondary-title></titles><pages>17-28</pages><dates><year>2001</year></dates><pub-location>Mahwah, NJ</pub-location><publisher>Lawrence Erlbaum</publisher><urls></urls></record></Cite></EndNote>(2001) argued that there is a need for L2 writing research to uncover students’ experiences in L2 English writing classes, “to hear their voices talking about the problems and successes they encountered in their writing classes and their interpretation of why things went as they did” (p. 17). Similarly, there is a need to listen to students’ voices in regards to assessment in these L2 writing classes. Rea-Dickins ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Rea-Dickins</Author><Year>1997</Year><RecNum>447</RecNum><DisplayText>(1997)</DisplayText><record><rec-number>447</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">447</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Rea-Dickins, Pauline</author></authors></contributors><titles><title>So, why do we need relationships with stakeholders in language testing? A view from the UK</title><secondary-title>Language Testing</secondary-title></titles><periodical><full-title>Language Testing</full-title></periodical><pages>304-314</pages><volume>14</volume><number>3</number><dates><year>1997</year><pub-dates><date>November 1, 1997</date></pub-dates></dates><urls><related-urls><url>;(1997) argued that the inclusion of learners’/ test-takers’ views of an assessment are particularly important to include because of the invaluable information they provide about how this group perceives a particular test task even though “their views are among the most difficult to make sense of and to use” (p. 306). Furthermore, the inclusion of the students’ voices can be expected to complement the teachers’ perspective in meaningful ways because these two different groups “will necessarily have different views on the need for or goals for assessment, the constructs to be measured in those assessments, and the benefit or harm to those being assessed” ADDIN EN.CITE <EndNote><Cite><Author>Baker</Author><Year>2010</Year><RecNum>444</RecNum><Pages>10</Pages><DisplayText>(Baker, 2010, p. 10)</DisplayText><record><rec-number>444</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">444</key></foreign-keys><ref-type name="Thesis">32</ref-type><contributors><authors><author>Baker, Beverly Anne</author></authors></contributors><titles><title>In the service of the stakeholder: A critical, mixed methods program of research in high stakes language assessment</title><secondary-title>McGill theses</secondary-title></titles><dates><year>2010</year></dates><pub-location>Montreal</pub-location><publisher>McGill University, Montreal, Quebec, Canada</publisher><work-type>Unpublished doctoral dissertation</work-type><urls><related-urls><url>;(Baker, 2010, p. 10). Consequently, this study also focuses on understanding what students know about the assessment process of grammatical ability in L2 writing and how the teachers’ assessment criteria might affect them. Focus of the StudyThe present study contributes to research in L2 writing assessment in the following ways. First, there has been little research on the assessment of grammar in the context of the classroom as opposed to high-stakes assessment on proficiency tests. Although we can draw on theoretical definitions of the construct of grammar for the purposes of language assessment from the literature, there is a need to examine the construct that teachers assess when they assign a grade or score for grammar on their students’ writing. Second, students’ point of view also needs to be considered as students provide an important perspective on how they are assessed by their teachers in L2 writing classrooms and on what the effect of the teacher’s assessment criteria is on the students’ writing and learning.MethodResearch Questions and DesignThe purpose of this study is to examine the following research questions:What indicators of grammatical ability do teachers attend to in their students’ L2 academic writing when assigning grades for grammar on an analytical rating scale?What knowledge do students have about the indicators their teachers look for when assigning a grade for grammar in writing?What do students perceive to be the impact of their teachers’ assessment criteria on their way of writing and learning in the L2 writing classroom?In a mixed methods triangulation research design ADDIN EN.CITE <EndNote><Cite><Author>Creswell</Author><Year>2007</Year><RecNum>187</RecNum><DisplayText>(Creswell & Plano Clark, 2007)</DisplayText><record><rec-number>187</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">187</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">691</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Creswell, John W.</author><author>Plano Clark, Vicki L.</author></authors></contributors><titles><title>Designing and conducting mixed methods research</title></titles><keywords><keyword>Social sciences Research Methodology.</keyword><keyword>Research Evaluation.</keyword></keywords><dates><year>2007</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>1412927919 (cloth)
1412927927 (pbk.)
9781412927918
9781412927925</isbn><urls><related-urls><url>Table of contents only description </url></related-urls></urls></record></Cite></EndNote>(Creswell & Plano Clark, 2007), qualitative (qual) and quantitative (quan) data sources were combined to answer these questions. Students’ essay exams were analyzed using quantitative text-based analysis measures to answer the first research question. In addition, student questionnaire and student- and teacher-interview data were analyzed qualitatively to answer the second and third research questions and to understand the context within which the exams took place. A case study approach was deemed most suitable for this study because it enabled the in-depth investigation of the assessment of grammar in L2 writing classrooms as a social phenomenon influenced by both teachers and students ADDIN EN.CITE <EndNote><Cite><Author>Yin</Author><Year>2009</Year><RecNum>370</RecNum><DisplayText>(Yin, 2009)</DisplayText><record><rec-number>370</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">370</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Yin, Robert K.</author></authors></contributors><titles><title>Case study research: Design and methods</title><secondary-title>Applied social research methods series ; 5</secondary-title></titles><edition>4th</edition><keywords><keyword>Case method.</keyword><keyword>Social sciences Research Methodology.</keyword></keywords><dates><year>2009</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>9781412960991 (pbk.)
1412960991 (pbk.)</isbn><urls></urls></record></Cite></EndNote>(Yin, 2009). Figure 1 provides an overview of the research design using Morse’s ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Huang</Author><Year>2010</Year><RecNum>443</RecNum><DisplayText>(2010)</DisplayText><record><rec-number>443</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">443</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Huang, Jinyan</author><author>Foote, Chandra J.</author></authors></contributors><titles><title>Grading between the lines: What really impacts professors’ holistic evaluation of ESL graduate student writing?</title><secondary-title>Language Assessment Quarterly</secondary-title></titles><periodical><full-title>Language Assessment Quarterly</full-title></periodical><pages>219-233</pages><volume>7</volume><number>3</number><dates><year>2010</year><pub-dates><date>2010/07/30</date></pub-dates></dates><publisher>Routledge</publisher><isbn>1543-4303</isbn><urls><related-urls><url>;(2010) notation for mixed methods research to indicate the relationship between data sources and analyses.[Insert Figure 1 about here]Research Context & ParticipantsThe research questions were investigated in a case study of two academic English as a second language (ESL) writing classes focusing on academic writing and vocabulary development at the high-intermediate (Course 1) and advanced level (Course 2) at an English-medium university in Canada. This institution admits students with a TOEFL iBT score of at least 75. If students score between 75 and 89, they may be required to take ESL writing courses, depending on their performance on the institutional placement test. At the same time, these students are already permitted to commence their discipline-specific academic programs. These L2 writing classes are general academic writing classes focusing on what Hyland ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Hyland</Author><Year>2006</Year><RecNum>450</RecNum><DisplayText>(2006)</DisplayText><record><rec-number>450</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">450</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Hyland, Ken</author></authors></contributors><titles><title>English for academic purposes: An advanced resource book</title></titles><dates><year>2006</year></dates><pub-location>London</pub-location><publisher>Routledge</publisher><urls></urls></record></Cite></EndNote>(2006) calls general English for Academic Purposes (EAP). Since the target students for these classes have only partially met the university’s proficiency requirement and therefore need further ESL training in an academic setting, students in these classes do not learn discipline-specific EAP or genres but develop their academic writing skills more generally, focusing on the “common core” ADDIN EN.CITE <EndNote><Cite><Author>Hyland</Author><Year>2006</Year><RecNum>450</RecNum><Pages>11</Pages><DisplayText>(Hyland, 2006, p. 11)</DisplayText><record><rec-number>450</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">450</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Hyland, Ken</author></authors></contributors><titles><title>English for academic purposes: An advanced resource book</title></titles><dates><year>2006</year></dates><pub-location>London</pub-location><publisher>Routledge</publisher><urls></urls></record></Cite></EndNote>(Hyland, 2006, p. 11) of academic writing skills (i.e., the common underlying language forms and skills) in a course with students from different faculties and programs. The approach to teaching writing is what has been called the post-process approach PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5GZXJyaXM8L0F1dGhvcj48WWVhcj4yMDA1PC9ZZWFyPjxS
ZWNOdW0+NDQ4PC9SZWNOdW0+PFByZWZpeD5lLmcuYCwgPC9QcmVmaXg+PERpc3BsYXlUZXh0Pihl
LmcuLCBGZXJyaXMgJmFtcDsgSGVkZ2NvY2ssIDIwMDU7IFBvbGlvICZhbXA7IFdpbGxpYW1zLCAy
MDA5KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj40NDg8L3JlYy1udW1iZXI+PGZv
cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVz
cGF4eHhyZnpmc3ciPjQ0ODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29r
Ij42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+RmVycmlzLCBEYW5h
IFIuPC9hdXRob3I+PGF1dGhvcj5IZWRnY29jaywgSm9objwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UZWFjaGluZyBFU0wgY29tcG9zaXRpb246IFB1cnBv
c2UsIHByb2Nlc3MsIGFuZCBwcmFjdGljZTwvdGl0bGU+PC90aXRsZXM+PG51bWJlcj54aXYsIDQz
MSBwLiA6PC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPkVuZ2xpc2ggbGFuZ3VhZ2UgU3R1ZHkg
YW5kIHRlYWNoaW5nIEZvcmVpZ24gc3BlYWtlcnMuPC9rZXl3b3JkPjxrZXl3b3JkPkVuZ2xpc2gg
bGFuZ3VhZ2UgQ29tcG9zaXRpb24gYW5kIGV4ZXJjaXNlcyBTdHVkeSBhbmQgdGVhY2hpbmcuPC9r
ZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMDU8L3llYXI+PC9kYXRlcz48cHViLWxv
Y2F0aW9uPk1haHdhaCwgTi5KLjsgTG9uZG9uPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5MYXdy
ZW5jZSBFcmxiYXVtPC9wdWJsaXNoZXI+PGlzYm4+MDgwNTg0NDY3OCAocGJrLiA6IGFsay4gcGFw
ZXIpPC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Qb2xp
bzwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJlY051bT40NDk8L1JlY051bT48cmVjb3JkPjxy
ZWMtbnVtYmVyPjQ0OTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGIt
aWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NDQ5PC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2sgU2VjdGlvbiI+NTwvcmVmLXR5cGU+PGNvbnRy
aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlBvbGlvLCBDaGFybGVuZSBHLjwvYXV0aG9yPjxhdXRo
b3I+V2lsbGlhbXMsIEplc3NpY2E8L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1hdXRob3Jz
PjxhdXRob3I+TG9uZywgTWljaGFlbCBILjwvYXV0aG9yPjxhdXRob3I+RG91Z2h0eSwgQ2F0aGVy
aW5lIEouPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5UZWFjaGluZyBhbmQgVGVzdGluZyBXcml0aW5nPC90aXRsZT48c2Vjb25kYXJ5LXRp
dGxlPlRoZSBIYW5kYm9vayBvZiBMYW5ndWFnZSBUZWFjaGluZzwvc2Vjb25kYXJ5LXRpdGxlPjwv
dGl0bGVzPjxwYWdlcz40ODYtNTE3PC9wYWdlcz48a2V5d29yZHM+PGtleXdvcmQ+dGVhY2hpbmcg
YW5kIHRlc3Rpbmcgd3JpdGluZzwva2V5d29yZD48a2V5d29yZD5zZWNvbmQvZm9yZWlnbiBsYW5n
dWFnZSAoTDIpIHdyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+Y29nbml0aXZlIGZhY3RvcnMgaW4g
bGVhcm5pbmcgdG8gd3JpdGU8L2tleXdvcmQ+PGtleXdvcmQ+U0xBIHJlc2VhcmNoLCBsYW5ndWFn
ZSB0ZWFjaGluZyBhbmQgcm9sZSBvZiBmb3JtLWZvY3VzZWQgaW5zdHJ1Y3Rpb248L2tleXdvcmQ+
PGtleXdvcmQ+ZXJyb3IgY29ycmVjdGlvbiBpbiB3cml0aW5nPC9rZXl3b3JkPjxrZXl3b3JkPndy
aXRpbmcgcHJvY2VzcyBhbmQgcHJvY2VzcyBhcHByb2FjaDwva2V5d29yZD48a2V5d29yZD5nZW5y
ZS1iYXNlZCBwZWRhZ29naWVzIGFuZCBpbmRpdmlkdWFsIHZvaWNlIGFuZCBzZWxmLWRpc2NvdmVy
eTwva2V5d29yZD48a2V5d29yZD5wcm9jZXNzIHBlZGFnb2dpZXMgLSBkb21pbmF0aW5nIE5vcnRo
IEFtZXJpY2FuIGNsYXNzcm9vbXM8L2tleXdvcmQ+PGtleXdvcmQ+Q29sbGVnZSBFbmdsaXNoIFRl
c3QgKENFVCkgYW5kIE5hdGlvbmFsIE1hdHJpY3VsYXRpb24gRW5nbGlzaCBUZXN0IChOTUVUKTwv
a2V5d29yZD48a2V5d29yZD5ldGhpY2FsIGlzc3VlcyByZWxhdGVkIHRvIEwyIGFzc2Vzc21lbnQ8
L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48L2RhdGVzPjxwdWJs
aXNoZXI+V2lsZXktQmxhY2t3ZWxsPC9wdWJsaXNoZXI+PGlzYm4+OTc4MTQ0NDMxNTc4MzwvaXNi
bj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAwMi85Nzgx
NDQ0MzE1NzgzLmNoMjY8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVz
b3VyY2UtbnVtPjEwLjEwMDIvOTc4MTQ0NDMxNTc4My5jaDI2PC9lbGVjdHJvbmljLXJlc291cmNl
LW51bT48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5GZXJyaXM8L0F1dGhvcj48WWVhcj4yMDA1PC9ZZWFyPjxS
ZWNOdW0+NDQ4PC9SZWNOdW0+PFByZWZpeD5lLmcuYCwgPC9QcmVmaXg+PERpc3BsYXlUZXh0Pihl
LmcuLCBGZXJyaXMgJmFtcDsgSGVkZ2NvY2ssIDIwMDU7IFBvbGlvICZhbXA7IFdpbGxpYW1zLCAy
MDA5KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj40NDg8L3JlYy1udW1iZXI+PGZv
cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVz
cGF4eHhyZnpmc3ciPjQ0ODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29r
Ij42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+RmVycmlzLCBEYW5h
IFIuPC9hdXRob3I+PGF1dGhvcj5IZWRnY29jaywgSm9objwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UZWFjaGluZyBFU0wgY29tcG9zaXRpb246IFB1cnBv
c2UsIHByb2Nlc3MsIGFuZCBwcmFjdGljZTwvdGl0bGU+PC90aXRsZXM+PG51bWJlcj54aXYsIDQz
MSBwLiA6PC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPkVuZ2xpc2ggbGFuZ3VhZ2UgU3R1ZHkg
YW5kIHRlYWNoaW5nIEZvcmVpZ24gc3BlYWtlcnMuPC9rZXl3b3JkPjxrZXl3b3JkPkVuZ2xpc2gg
bGFuZ3VhZ2UgQ29tcG9zaXRpb24gYW5kIGV4ZXJjaXNlcyBTdHVkeSBhbmQgdGVhY2hpbmcuPC9r
ZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMDU8L3llYXI+PC9kYXRlcz48cHViLWxv
Y2F0aW9uPk1haHdhaCwgTi5KLjsgTG9uZG9uPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5MYXdy
ZW5jZSBFcmxiYXVtPC9wdWJsaXNoZXI+PGlzYm4+MDgwNTg0NDY3OCAocGJrLiA6IGFsay4gcGFw
ZXIpPC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Qb2xp
bzwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJlY051bT40NDk8L1JlY051bT48cmVjb3JkPjxy
ZWMtbnVtYmVyPjQ0OTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGIt
aWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NDQ5PC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2sgU2VjdGlvbiI+NTwvcmVmLXR5cGU+PGNvbnRy
aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlBvbGlvLCBDaGFybGVuZSBHLjwvYXV0aG9yPjxhdXRo
b3I+V2lsbGlhbXMsIEplc3NpY2E8L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1hdXRob3Jz
PjxhdXRob3I+TG9uZywgTWljaGFlbCBILjwvYXV0aG9yPjxhdXRob3I+RG91Z2h0eSwgQ2F0aGVy
aW5lIEouPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5UZWFjaGluZyBhbmQgVGVzdGluZyBXcml0aW5nPC90aXRsZT48c2Vjb25kYXJ5LXRp
dGxlPlRoZSBIYW5kYm9vayBvZiBMYW5ndWFnZSBUZWFjaGluZzwvc2Vjb25kYXJ5LXRpdGxlPjwv
dGl0bGVzPjxwYWdlcz40ODYtNTE3PC9wYWdlcz48a2V5d29yZHM+PGtleXdvcmQ+dGVhY2hpbmcg
YW5kIHRlc3Rpbmcgd3JpdGluZzwva2V5d29yZD48a2V5d29yZD5zZWNvbmQvZm9yZWlnbiBsYW5n
dWFnZSAoTDIpIHdyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+Y29nbml0aXZlIGZhY3RvcnMgaW4g
bGVhcm5pbmcgdG8gd3JpdGU8L2tleXdvcmQ+PGtleXdvcmQ+U0xBIHJlc2VhcmNoLCBsYW5ndWFn
ZSB0ZWFjaGluZyBhbmQgcm9sZSBvZiBmb3JtLWZvY3VzZWQgaW5zdHJ1Y3Rpb248L2tleXdvcmQ+
PGtleXdvcmQ+ZXJyb3IgY29ycmVjdGlvbiBpbiB3cml0aW5nPC9rZXl3b3JkPjxrZXl3b3JkPndy
aXRpbmcgcHJvY2VzcyBhbmQgcHJvY2VzcyBhcHByb2FjaDwva2V5d29yZD48a2V5d29yZD5nZW5y
ZS1iYXNlZCBwZWRhZ29naWVzIGFuZCBpbmRpdmlkdWFsIHZvaWNlIGFuZCBzZWxmLWRpc2NvdmVy
eTwva2V5d29yZD48a2V5d29yZD5wcm9jZXNzIHBlZGFnb2dpZXMgLSBkb21pbmF0aW5nIE5vcnRo
IEFtZXJpY2FuIGNsYXNzcm9vbXM8L2tleXdvcmQ+PGtleXdvcmQ+Q29sbGVnZSBFbmdsaXNoIFRl
c3QgKENFVCkgYW5kIE5hdGlvbmFsIE1hdHJpY3VsYXRpb24gRW5nbGlzaCBUZXN0IChOTUVUKTwv
a2V5d29yZD48a2V5d29yZD5ldGhpY2FsIGlzc3VlcyByZWxhdGVkIHRvIEwyIGFzc2Vzc21lbnQ8
L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48L2RhdGVzPjxwdWJs
aXNoZXI+V2lsZXktQmxhY2t3ZWxsPC9wdWJsaXNoZXI+PGlzYm4+OTc4MTQ0NDMxNTc4MzwvaXNi
bj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAwMi85Nzgx
NDQ0MzE1NzgzLmNoMjY8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVz
b3VyY2UtbnVtPjEwLjEwMDIvOTc4MTQ0NDMxNTc4My5jaDI2PC9lbGVjdHJvbmljLXJlc291cmNl
LW51bT48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE.DATA (e.g., Ferris & Hedgcock, 2005; Polio & Williams, 2009). That is to say, there is a focus on the cognitive processes involved in the production, revision, and editing of a piece of writing; at the same time, the product of the writing process is important. Essays are the primary focus in these classes, and students produce two drafts of all assignments written in class or at home. The focus of analysis for this study is the midterm essay exam, which is a task similar to class essay assignments and the final essay exam and serves both formative and summative purposes in these classes: formative by providing feedback to the students so that they can continue to improve for the remainder of the course and summative by measuring student achievement at the midpoint of the course and contributing 20% to the students’ course grade. In the ESL program where the study took place, teachers evaluate their own students’ essay exams. However, in order to be accountable to students and ensure similar standards of assessment across different sections or classes of the same course, the essays with the highest and lowest scores are also read by another teacher teaching the same course. The number of essays that are re-read for each class varies from course to course but typically ranges from 10 to 30%. For essay exams, all teachers have to use the ESL program evaluation grid (see Appendix A).Thirty-three students in the two courses and their teachers participated in this study. All student participants were in the first two years of their degree, and half were in their first or second semester. All except one student were enrolled in an undergraduate program. Table 1 presents detailed background information on the student participants. As is evident, the student population in the two classes is comparable in terms of gender distribution, mean age, L1 backgrounds, and major programs. The two teachers were native speakers of English with degrees in teaching ESL and 10 and 33 years of experience teaching and assessing ESL.[Insert Table 1 about here]InstrumentsThe student questionnaire (see Appendix B) was developed specifically for this study. The first part of the questionnaire was designed to obtain demographic data to describe the student population in this study. Most questions in the second part of the questionnaire were open-ended and served to answer the research questions as well as identify themes for the student interviews.For the interviews, specific interview protocols were developed. For the student interviews, the questions were based on part 2 of the student questionnaire and were designed as follow-up and in-depth questions (see Appendix C). The teacher interview questions were formulated based on research and theory questions (see Appendix D). A detailed discussion and analysis of particular essays with the teacher participants was included as an interview intervention to clarify the teachers’ assessment criteria for grammatical ability. For this purpose, four essays that represented different levels of performance on the grammar grade were selected for each teacher to stimulate the discussion. ProceduresThe study took place during one 13-week semester. At the end of week 7, the students wrote their midterm essay exam. In both classes, students could choose one of three topics that elicited the same rhetorical pattern (see Appendix E for the list of topics) and had 2.5 hours to write the essay. Before teachers marked the exams, the exams were photocopied for the text-based analysis (Phase 1 of the study, see Figure 1). After the teachers had assigned grades, the completed evaluation grid (see Appendix A) for each exam was photocopied. The teachers then returned the exams and evaluation grids to the students. In the following class, the student questionnaire was administered (Phase 2 of the study, see Figure 1). This ensured that students had had time to review their midterm exam and reflect on their grammar mark. Towards the end of the semester, the eight students who had agreed to be interviewed and have their interviews recorded and the two teachers were interviewed (Phase 3 of the study, see Figure 1). All interviews were recorded and then transcribed. Both teachers and students were compensated financially for their time to conduct the interviews.Data AnalysisPhase 1: Text-based analysis of exams (data to answer research question 1).For the text-based analysis, first all production units (clauses, sentences, and T-units) were hand-tagged on all exams and then tallied by the researcher. Sentences were defined by the writer’s punctuation as phrases that are intercalated between two termination marks (i.e., periods, questions marks, or exclamation marks) and start with a capital letter PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5IdW50PC9BdXRob3I+PFllYXI+MTk2NTwvWWVhcj48UmVj
TnVtPjEyNjwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oSGFsbGlkYXkgJmFtcDsgTWF0dGhpZXNzZW4s
IDIwMDQ7IEhvbWJ1cmcsIDE5ODQ7IEh1bnQsIDE5NjU7IFRhcGlhLCAxOTkzKTwvRGlzcGxheVRl
eHQ+PHJlY29yZD48cmVjLW51bWJlcj4xMjY8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjEy
Njwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjYzNjwv
a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29u
dHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+SHVudCwgSy4gVy48L2F1dGhvcj48L2F1dGhvcnM+
PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+R3JhbW1hdGljYWwgc3RydWN0dXJlcyB3cml0
dGVuIGF0IHRocmVlIGdyYWRlIGxldmVsczwvdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE1OTwvcGFn
ZXM+PGRhdGVzPjx5ZWFyPjE5NjU8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPlVyYmFuYSwg
SUw8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlRoZSBOYXRpb25hbCBDb3VuY2lsIG9mIFRlYWNo
ZXJzIG9mIEVuZ2xpc2g8L3B1Ymxpc2hlcj48Y2FsbC1udW0+UEUxMDExIE4yOTUgbm8uIDMgPC9j
YWxsLW51bT48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+SGFsbGlk
YXk8L0F1dGhvcj48WWVhcj4yMDA0PC9ZZWFyPjxSZWNOdW0+MTk5PC9SZWNOdW0+PHJlY29yZD48
cmVjLW51bWJlcj4xOTk8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRi
LWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjE5OTwva2V5PjxrZXkg
YXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjcwMzwva2V5PjwvZm9yZWln
bi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxh
dXRob3JzPjxhdXRob3I+SGFsbGlkYXksIE0uIEEuIEsuPC9hdXRob3I+PGF1dGhvcj5NYXR0aGll
c3NlbiwgQ2hyaXN0aWFuIE0uIEkuIE0uPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPkFuIGludHJvZHVjdGlvbiB0byBmdW5jdGlvbmFsIGdyYW1tYXI8L3Rp
dGxlPjwvdGl0bGVzPjxlZGl0aW9uPjNyZDwvZWRpdGlvbj48ZGF0ZXM+PHllYXI+MjAwNDwveWVh
cj48L2RhdGVzPjxwdWItbG9jYXRpb24+TG9uZG9uPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5B
cm5vbGQ8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+SG9tYnVyZzwvQXV0aG9yPjxZZWFyPjE5ODQ8L1llYXI+PFJlY051bT42MzwvUmVjTnVtPjxy
ZWNvcmQ+PHJlYy1udW1iZXI+NjM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i
RU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjYzPC9rZXk+
PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTc4PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Ib21idXJnLCBUYWNvIEp1c3R1czwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Ib2xpc3RpYyBldmFs
dWF0aW9uIG9mIEVTTCBjb21wb3NpdGlvbnM6IENhbiBpdCBiZSB2YWxpZGF0ZWQgb2JqZWN0aXZl
bHk/PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlRFU09MIFF1YXJ0ZXJseTwvc2Vjb25kYXJ5LXRp
dGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPlRFU09MIFF1YXJ0ZXJseTwvZnVs
bC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjg3LTEwNzwvcGFnZXM+PHZvbHVtZT4xODwvdm9s
dW1lPjxudW1iZXI+MTwvbnVtYmVyPjxkYXRlcz48eWVhcj4xOTg0PC95ZWFyPjwvZGF0ZXM+PHVy
bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlRhcGlhPC9BdXRob3I+PFll
YXI+MTk5MzwvWWVhcj48UmVjTnVtPjEwOTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTA5
PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2
d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4xMDk8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRi
LWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj42MjA8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5
cGUgbmFtZT0iVGhlc2lzIj4zMjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPlRhcGlhLCBFbGVuYTwvYXV0aG9yPjwvYXV0aG9ycz48dGVydGlhcnktYXV0aG9ycz48YXV0
aG9yPkJhcmRvdmktSGFybGlnLCBLYXRobGVlbjwvYXV0aG9yPjwvdGVydGlhcnktYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Db2duaXRpdmUgZGVtYW5kcyBhcyBhIGZhY3Rv
ciBpbiBpbnRlcmxhbmd1YWdlIHN5bnRheDogQSBzdHVkeSBpbiB0b3BpY3MgYW5kIHRleHRzPC90
aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkRlcGFydG1lbnQgb2YgTGluZ3Vpc3RpY3M8L3NlY29uZGFy
eS10aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MTk5MzwveWVhcj48L2RhdGVzPjxwdWItbG9j
YXRpb24+Qmxvb21pbmd0b248L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPkluZGlhbmEgVW5pdmVy
c2l0eTwvcHVibGlzaGVyPjx3b3JrLXR5cGU+VW5wdWJsaXNoZWQgZG9jdG9yYWwgZGlzc2VydGF0
aW9uPC93b3JrLXR5cGU+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly9wcm9xdWVzdC51
bWkuY29tL3BxZHdlYj9kaWQ9NzQ2MzE1ODcxJmFtcDtzaWQ9MyZhbXA7Rm10PTImYW1wO2NsaWVu
dElkPTEwODQzJmFtcDtSUVQ9MzA5JmFtcDtWTmFtZT1QUUQ8L3VybD48L3JlbGF0ZWQtdXJscz48
L3VybHM+PGFjY2Vzcy1kYXRlPk5vdmVtYmVyIDEwLCAyMDA2PC9hY2Nlc3MtZGF0ZT48L3JlY29y
ZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5IdW50PC9BdXRob3I+PFllYXI+MTk2NTwvWWVhcj48UmVj
TnVtPjEyNjwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oSGFsbGlkYXkgJmFtcDsgTWF0dGhpZXNzZW4s
IDIwMDQ7IEhvbWJ1cmcsIDE5ODQ7IEh1bnQsIDE5NjU7IFRhcGlhLCAxOTkzKTwvRGlzcGxheVRl
eHQ+PHJlY29yZD48cmVjLW51bWJlcj4xMjY8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjEy
Njwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjYzNjwv
a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29u
dHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+SHVudCwgSy4gVy48L2F1dGhvcj48L2F1dGhvcnM+
PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+R3JhbW1hdGljYWwgc3RydWN0dXJlcyB3cml0
dGVuIGF0IHRocmVlIGdyYWRlIGxldmVsczwvdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE1OTwvcGFn
ZXM+PGRhdGVzPjx5ZWFyPjE5NjU8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPlVyYmFuYSwg
SUw8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlRoZSBOYXRpb25hbCBDb3VuY2lsIG9mIFRlYWNo
ZXJzIG9mIEVuZ2xpc2g8L3B1Ymxpc2hlcj48Y2FsbC1udW0+UEUxMDExIE4yOTUgbm8uIDMgPC9j
YWxsLW51bT48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+SGFsbGlk
YXk8L0F1dGhvcj48WWVhcj4yMDA0PC9ZZWFyPjxSZWNOdW0+MTk5PC9SZWNOdW0+PHJlY29yZD48
cmVjLW51bWJlcj4xOTk8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRi
LWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjE5OTwva2V5PjxrZXkg
YXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjcwMzwva2V5PjwvZm9yZWln
bi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxh
dXRob3JzPjxhdXRob3I+SGFsbGlkYXksIE0uIEEuIEsuPC9hdXRob3I+PGF1dGhvcj5NYXR0aGll
c3NlbiwgQ2hyaXN0aWFuIE0uIEkuIE0uPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPkFuIGludHJvZHVjdGlvbiB0byBmdW5jdGlvbmFsIGdyYW1tYXI8L3Rp
dGxlPjwvdGl0bGVzPjxlZGl0aW9uPjNyZDwvZWRpdGlvbj48ZGF0ZXM+PHllYXI+MjAwNDwveWVh
cj48L2RhdGVzPjxwdWItbG9jYXRpb24+TG9uZG9uPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5B
cm5vbGQ8L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+SG9tYnVyZzwvQXV0aG9yPjxZZWFyPjE5ODQ8L1llYXI+PFJlY051bT42MzwvUmVjTnVtPjxy
ZWNvcmQ+PHJlYy1udW1iZXI+NjM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i
RU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjYzPC9rZXk+
PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTc4PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Ib21idXJnLCBUYWNvIEp1c3R1czwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Ib2xpc3RpYyBldmFs
dWF0aW9uIG9mIEVTTCBjb21wb3NpdGlvbnM6IENhbiBpdCBiZSB2YWxpZGF0ZWQgb2JqZWN0aXZl
bHk/PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlRFU09MIFF1YXJ0ZXJseTwvc2Vjb25kYXJ5LXRp
dGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPlRFU09MIFF1YXJ0ZXJseTwvZnVs
bC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjg3LTEwNzwvcGFnZXM+PHZvbHVtZT4xODwvdm9s
dW1lPjxudW1iZXI+MTwvbnVtYmVyPjxkYXRlcz48eWVhcj4xOTg0PC95ZWFyPjwvZGF0ZXM+PHVy
bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlRhcGlhPC9BdXRob3I+PFll
YXI+MTk5MzwvWWVhcj48UmVjTnVtPjEwOTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTA5
PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2
d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4xMDk8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRi
LWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj42MjA8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5
cGUgbmFtZT0iVGhlc2lzIj4zMjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPlRhcGlhLCBFbGVuYTwvYXV0aG9yPjwvYXV0aG9ycz48dGVydGlhcnktYXV0aG9ycz48YXV0
aG9yPkJhcmRvdmktSGFybGlnLCBLYXRobGVlbjwvYXV0aG9yPjwvdGVydGlhcnktYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Db2duaXRpdmUgZGVtYW5kcyBhcyBhIGZhY3Rv
ciBpbiBpbnRlcmxhbmd1YWdlIHN5bnRheDogQSBzdHVkeSBpbiB0b3BpY3MgYW5kIHRleHRzPC90
aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkRlcGFydG1lbnQgb2YgTGluZ3Vpc3RpY3M8L3NlY29uZGFy
eS10aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MTk5MzwveWVhcj48L2RhdGVzPjxwdWItbG9j
YXRpb24+Qmxvb21pbmd0b248L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPkluZGlhbmEgVW5pdmVy
c2l0eTwvcHVibGlzaGVyPjx3b3JrLXR5cGU+VW5wdWJsaXNoZWQgZG9jdG9yYWwgZGlzc2VydGF0
aW9uPC93b3JrLXR5cGU+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly9wcm9xdWVzdC51
bWkuY29tL3BxZHdlYj9kaWQ9NzQ2MzE1ODcxJmFtcDtzaWQ9MyZhbXA7Rm10PTImYW1wO2NsaWVu
dElkPTEwODQzJmFtcDtSUVQ9MzA5JmFtcDtWTmFtZT1QUUQ8L3VybD48L3JlbGF0ZWQtdXJscz48
L3VybHM+PGFjY2Vzcy1kYXRlPk5vdmVtYmVyIDEwLCAyMDA2PC9hY2Nlc3MtZGF0ZT48L3JlY29y
ZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Halliday & Matthiessen, 2004; Homburg, 1984; Hunt, 1965; Tapia, 1993). T-units were defined as one independent clause with all its dependent clauses ADDIN EN.CITE <EndNote><Cite><Author>Hunt</Author><Year>1965</Year><RecNum>126</RecNum><DisplayText>(Hunt, 1965)</DisplayText><record><rec-number>126</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">126</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">636</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Hunt, K. W.</author></authors></contributors><titles><title>Grammatical structures written at three grade levels</title></titles><pages>159</pages><dates><year>1965</year></dates><pub-location>Urbana, IL</pub-location><publisher>The National Council of Teachers of English</publisher><call-num>PE1011 N295 no. 3 </call-num><urls></urls></record></Cite></EndNote>(Hunt, 1965). Following and accepting the writer’s punctuation when coding sentences and T-units meant that this sometimes led to errors in syntax (incomplete sentences or independent clauses not separated/joined appropriately). Quirk, Greenbaum, Leech, and Svartvik’s ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Quirk</Author><Year>1985</Year><RecNum>219</RecNum><DisplayText>(1985)</DisplayText><record><rec-number>219</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">219</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">722</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Quirk, Randolph</author><author>Greenbaum, Sidney</author><author>Leech, Geoffrey</author><author>Svartvik, Jan</author></authors></contributors><titles><title>A comprehensive grammar of the English language</title></titles><keywords><keyword>English language Grammar.</keyword></keywords><dates><year>1985</year></dates><pub-location>London & New York</pub-location><publisher>Longman</publisher><isbn>0582517346 :</isbn><call-num>PE 1106 C65 1985</call-num><urls></urls></record></Cite></EndNote>(1985) definition of sentence limits as “wherever grammatical relations (such as those of subordination and coordination) cannot be established between clauses” (p. 48) was used to determine when the writer’s erroneous punctuation led to syntactic errors. For clauses, Quirk et al.’s ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Quirk</Author><Year>1985</Year><RecNum>219</RecNum><DisplayText>(1985)</DisplayText><record><rec-number>219</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">219</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">722</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Quirk, Randolph</author><author>Greenbaum, Sidney</author><author>Leech, Geoffrey</author><author>Svartvik, Jan</author></authors></contributors><titles><title>A comprehensive grammar of the English language</title></titles><keywords><keyword>English language Grammar.</keyword></keywords><dates><year>1985</year></dates><pub-location>London & New York</pub-location><publisher>Longman</publisher><isbn>0582517346 :</isbn><call-num>PE 1106 C65 1985</call-num><urls></urls></record></Cite></EndNote>(1985) definition was adopted with two revisions: (a) all clauses consist of at least one finite or non-finite verb (i.e., verbless clauses were excluded) and (b) all verbs had to be accompanied by at least one other constituent. If verbs did not have a dependent constituent, they were considered part of another clause. To assess the L2 writers’ ability to write accurately, morphological and syntactic errors were hand-tagged and tallied in order to calculate the errors per 100 words ratio. This measure is less commonly used than the errors per T-unit ratio ADDIN EN.CITE <EndNote><Cite><Author>Wolfe-Quintero</Author><Year>1998</Year><RecNum>13</RecNum><DisplayText>(Wolfe-Quintero et al., 1998)</DisplayText><record><rec-number>13</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">13</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">538</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Wolfe-Quintero, Kate</author><author>Inagaki, Shunji</author><author>Kim, Hae-Young</author></authors></contributors><titles><title>Second language development in writing: Measures of fluency, accuracy & complexity</title></titles><pages>187</pages><dates><year>1998</year></dates><pub-location>Honolulu, HI</pub-location><publisher>University of Hawai'i, Second Language Teaching and Curriculum Center</publisher><isbn>17</isbn><call-num>P53 W65 1998</call-num><work-type>Technical Report</work-type><urls></urls></record></Cite></EndNote>(Wolfe-Quintero et al., 1998), but it provides a more precise measure of the level of accuracy in L2 texts. Following Bardovi-Harlig and Bofman’s ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Bardovi-Harlig</Author><Year>1989</Year><RecNum>33</RecNum><DisplayText>(1989)</DisplayText><record><rec-number>33</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">33</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">553</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Bardovi-Harlig, Kathleen</author><author>Bofman, Theodora</author></authors></contributors><titles><title>Attainment of syntactic and morphological accuracy by advanced language learners</title><secondary-title>Studies in Second Language Acquisition</secondary-title></titles><periodical><full-title>Studies in Second Language Acquisition</full-title></periodical><pages>17-34</pages><volume>11</volume><number>1</number><dates><year>1989</year></dates><urls></urls><electronic-resource-num>10.1017/S0272263100007816</electronic-resource-num></record></Cite></EndNote>(1989) definition, morphological errors were defined as “errors in nominal morphology (plural, number agreement, and possessive), errors in verbal morphology (tense, subject-verb agreement, and passive formation), errors in determiners and articles, and errors in prepositions” (p. 21). Syntactic errors “consisted of word order errors, errors resulting from the absence [or addition] of major and minor constituents, and errors in combining sentences” ADDIN EN.CITE <EndNote><Cite ExcludeYear="1"><Author>Bardovi-Harlig</Author><Year>1989</Year><RecNum>33</RecNum><Pages>21</Pages><DisplayText>(Bardovi-Harlig & Bofman, p. 21)</DisplayText><record><rec-number>33</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">33</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">553</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Bardovi-Harlig, Kathleen</author><author>Bofman, Theodora</author></authors></contributors><titles><title>Attainment of syntactic and morphological accuracy by advanced language learners</title><secondary-title>Studies in Second Language Acquisition</secondary-title></titles><periodical><full-title>Studies in Second Language Acquisition</full-title></periodical><pages>17-34</pages><volume>11</volume><number>1</number><dates><year>1989</year></dates><urls></urls><electronic-resource-num>10.1017/S0272263100007816</electronic-resource-num></record></Cite></EndNote>(Bardovi-Harlig & Bofman, p. 21). Spelling and punctuation were ignored unless they led to morphologic errors (i.e., wrong verb or plural form) or syntactic errors (i.e., incomplete sentences).Following Norris and Ortega’s ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Norris</Author><Year>2009</Year><RecNum>366</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>366</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">366</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Norris, John M.</author><author>Ortega, Lourdes</author></authors></contributors><titles><title>Towards an organic approach to investigating CAF in instructed SLA: The case of complexity</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>555-578</pages><volume>30</volume><number>4</number><dates><year>2009</year><pub-dates><date>December 1, 2009</date></pub-dates></dates><urls><related-urls><url>;(2009) suggestion for a multidimensional definition of complexity, two complexity measures that gauge the level of subordination and phrasal elaboration were select: the clauses per T-unit ratio and the words per clause ratio. The clauses per T-unit ratio was chosen because it is a measure of subordination rather than coordination and therefore is more appropriate for higher-level L2 writers ADDIN EN.CITE <EndNote><Cite><Author>Norris</Author><Year>2009</Year><RecNum>366</RecNum><DisplayText>(Norris & Ortega, 2009)</DisplayText><record><rec-number>366</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">366</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Norris, John M.</author><author>Ortega, Lourdes</author></authors></contributors><titles><title>Towards an organic approach to investigating CAF in instructed SLA: The case of complexity</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>555-578</pages><volume>30</volume><number>4</number><dates><year>2009</year><pub-dates><date>December 1, 2009</date></pub-dates></dates><urls><related-urls><url>;(Norris & Ortega, 2009). Furthermore, it is the most commonly used subordination measure and has been found in previous research to increase as the writers’ proficiency level increases ADDIN EN.CITE <EndNote><Cite><Author>Ortega</Author><Year>2003</Year><RecNum>66</RecNum><DisplayText>(Ortega, 2003)</DisplayText><record><rec-number>66</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">66</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">581</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Ortega, Lourdes</author></authors></contributors><titles><title>Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>492-518</pages><volume>24</volume><number>4</number><dates><year>2003</year></dates><urls><related-urls><url> </electronic-resource-num></record></Cite></EndNote>(Ortega, 2003), albeit not always with statistical significance ADDIN EN.CITE <EndNote><Cite><Author>Wolfe-Quintero</Author><Year>1998</Year><RecNum>13</RecNum><DisplayText>(Wolfe-Quintero et al., 1998)</DisplayText><record><rec-number>13</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">13</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">538</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Wolfe-Quintero, Kate</author><author>Inagaki, Shunji</author><author>Kim, Hae-Young</author></authors></contributors><titles><title>Second language development in writing: Measures of fluency, accuracy & complexity</title></titles><pages>187</pages><dates><year>1998</year></dates><pub-location>Honolulu, HI</pub-location><publisher>University of Hawai'i, Second Language Teaching and Curriculum Center</publisher><isbn>17</isbn><call-num>P53 W65 1998</call-num><work-type>Technical Report</work-type><urls></urls></record></Cite></EndNote>(Wolfe-Quintero et al., 1998). The words per clause ratio measure was chosen because it assesses to what extent L2 writers use subclausal structures such as adjectives, adverbs, and prepositional phrases to increase clause length ADDIN EN.CITE <EndNote><Cite><Author>Norris</Author><Year>2009</Year><RecNum>366</RecNum><DisplayText>(Norris & Ortega, 2009)</DisplayText><record><rec-number>366</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">366</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Norris, John M.</author><author>Ortega, Lourdes</author></authors></contributors><titles><title>Towards an organic approach to investigating CAF in instructed SLA: The case of complexity</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>555-578</pages><volume>30</volume><number>4</number><dates><year>2009</year><pub-dates><date>December 1, 2009</date></pub-dates></dates><urls><related-urls><url>;(Norris & Ortega, 2009). The combination of these two measures is useful because two different patterns of complexification are hypothesized ADDIN EN.CITE <EndNote><Cite><Author>Halliday</Author><Year>1993</Year><RecNum>491</RecNum><DisplayText>(Halliday & Martin, 1993)</DisplayText><record><rec-number>491</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">491</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Halliday, M. A. K.</author><author>Martin, J. R.</author></authors></contributors><titles><title>Writing science: Literacy and discursive power</title><secondary-title>Pittsburgh series in composition, literacy, and culture</secondary-title></titles><number>xii, 283 p. :</number><keywords><keyword>Technical writing.</keyword><keyword>Communication in science.</keyword></keywords><dates><year>1993</year></dates><pub-location>Pittsburgh</pub-location><publisher>University of Pittsburgh Press</publisher><isbn>0822911809 (cl)
0822961032 (pbk)</isbn><urls></urls></record></Cite></EndNote>(Halliday & Martin, 1993) and observed with higher-level learners: L2 writers first expand their writing through subordination PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5PcnRlZ2E8L0F1dGhvcj48WWVhcj4yMDAzPC9ZZWFyPjxS
ZWNOdW0+NjY8L1JlY051bT48UHJlZml4PmEgcGF0dGVybiBvYnNlcnZlZCBpbiBtYW55IEwyIHdy
aXRpbmcgc3R1ZGllcyB3aXRoIGludGVybWVkaWF0ZS1sZXZlbCB3cml0ZXJzYDsgc2VlYCwgZm9y
IGV4YW1wbGVgLCA8L1ByZWZpeD48RGlzcGxheVRleHQ+KGEgcGF0dGVybiBvYnNlcnZlZCBpbiBt
YW55IEwyIHdyaXRpbmcgc3R1ZGllcyB3aXRoIGludGVybWVkaWF0ZS1sZXZlbCB3cml0ZXJzOyBz
ZWUsIGZvciBleGFtcGxlLCBPcnRlZ2EsIDIwMDM7IFdvbGZlLVF1aW50ZXJvIGV0IGFsLiwgMTk5
OCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+NjY8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4
eHhyZnpmc3ciPjY2PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFw
dk1DRSI+NTgxPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0
aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5PcnRlZ2Es
IExvdXJkZXM8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+
U3ludGFjdGljIGNvbXBsZXhpdHkgbWVhc3VyZXMgYW5kIHRoZWlyIHJlbGF0aW9uc2hpcCB0byBM
MiBwcm9maWNpZW5jeTogQSByZXNlYXJjaCBzeW50aGVzaXMgb2YgY29sbGVnZS1sZXZlbCBMMiB3
cml0aW5nPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L3NlY29u
ZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5BcHBsaWVkIExpbmd1
aXN0aWNzPC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NDkyLTUxODwvcGFnZXM+PHZv
bHVtZT4yNDwvdm9sdW1lPjxudW1iZXI+NDwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDAzPC95ZWFy
PjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly9hcHBsaWoub3hmb3Jkam91
cm5hbHMub3JnL2NvbnRlbnQvMjQvNC80OTIuZnVsbC5wZGYraHRtbDwvdXJsPjwvcmVsYXRlZC11
cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA5My9hcHBsaW4vMjQuNC40
OTIgPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhv
cj5Xb2xmZS1RdWludGVybzwvQXV0aG9yPjxZZWFyPjE5OTg8L1llYXI+PFJlY051bT4xMzwvUmVj
TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjEz
PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM4PC9r
ZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Xb2xmZS1RdWludGVybywgS2F0ZTwvYXV0aG9yPjxh
dXRob3I+SW5hZ2FraSwgU2h1bmppPC9hdXRob3I+PGF1dGhvcj5LaW0sIEhhZS1Zb3VuZzwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5TZWNvbmQgbGFuZ3Vh
Z2UgZGV2ZWxvcG1lbnQgaW4gd3JpdGluZzogTWVhc3VyZXMgb2YgZmx1ZW5jeSwgYWNjdXJhY3kg
JmFtcDsgY29tcGxleGl0eTwvdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE4NzwvcGFnZXM+PGRhdGVz
Pjx5ZWFyPjE5OTg8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkhvbm9sdWx1LCBISTwvcHVi
LWxvY2F0aW9uPjxwdWJsaXNoZXI+VW5pdmVyc2l0eSBvZiBIYXdhaSZhcG9zO2ksIFNlY29uZCBM
YW5ndWFnZSBUZWFjaGluZyBhbmQgQ3VycmljdWx1bSBDZW50ZXI8L3B1Ymxpc2hlcj48aXNibj4x
NzwvaXNibj48Y2FsbC1udW0+UDUzIFc2NSAxOTk4PC9jYWxsLW51bT48d29yay10eXBlPlRlY2hu
aWNhbCBSZXBvcnQ8L3dvcmstdHlwZT48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5k
Tm90ZT5=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5PcnRlZ2E8L0F1dGhvcj48WWVhcj4yMDAzPC9ZZWFyPjxS
ZWNOdW0+NjY8L1JlY051bT48UHJlZml4PmEgcGF0dGVybiBvYnNlcnZlZCBpbiBtYW55IEwyIHdy
aXRpbmcgc3R1ZGllcyB3aXRoIGludGVybWVkaWF0ZS1sZXZlbCB3cml0ZXJzYDsgc2VlYCwgZm9y
IGV4YW1wbGVgLCA8L1ByZWZpeD48RGlzcGxheVRleHQ+KGEgcGF0dGVybiBvYnNlcnZlZCBpbiBt
YW55IEwyIHdyaXRpbmcgc3R1ZGllcyB3aXRoIGludGVybWVkaWF0ZS1sZXZlbCB3cml0ZXJzOyBz
ZWUsIGZvciBleGFtcGxlLCBPcnRlZ2EsIDIwMDM7IFdvbGZlLVF1aW50ZXJvIGV0IGFsLiwgMTk5
OCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+NjY8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4
eHhyZnpmc3ciPjY2PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFw
dk1DRSI+NTgxPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0
aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5PcnRlZ2Es
IExvdXJkZXM8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+
U3ludGFjdGljIGNvbXBsZXhpdHkgbWVhc3VyZXMgYW5kIHRoZWlyIHJlbGF0aW9uc2hpcCB0byBM
MiBwcm9maWNpZW5jeTogQSByZXNlYXJjaCBzeW50aGVzaXMgb2YgY29sbGVnZS1sZXZlbCBMMiB3
cml0aW5nPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L3NlY29u
ZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5BcHBsaWVkIExpbmd1
aXN0aWNzPC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NDkyLTUxODwvcGFnZXM+PHZv
bHVtZT4yNDwvdm9sdW1lPjxudW1iZXI+NDwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDAzPC95ZWFy
PjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly9hcHBsaWoub3hmb3Jkam91
cm5hbHMub3JnL2NvbnRlbnQvMjQvNC80OTIuZnVsbC5wZGYraHRtbDwvdXJsPjwvcmVsYXRlZC11
cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA5My9hcHBsaW4vMjQuNC40
OTIgPC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhv
cj5Xb2xmZS1RdWludGVybzwvQXV0aG9yPjxZZWFyPjE5OTg8L1llYXI+PFJlY051bT4xMzwvUmVj
TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjEz
PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1DRSI+NTM4PC9r
ZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Xb2xmZS1RdWludGVybywgS2F0ZTwvYXV0aG9yPjxh
dXRob3I+SW5hZ2FraSwgU2h1bmppPC9hdXRob3I+PGF1dGhvcj5LaW0sIEhhZS1Zb3VuZzwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5TZWNvbmQgbGFuZ3Vh
Z2UgZGV2ZWxvcG1lbnQgaW4gd3JpdGluZzogTWVhc3VyZXMgb2YgZmx1ZW5jeSwgYWNjdXJhY3kg
JmFtcDsgY29tcGxleGl0eTwvdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE4NzwvcGFnZXM+PGRhdGVz
Pjx5ZWFyPjE5OTg8L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPkhvbm9sdWx1LCBISTwvcHVi
LWxvY2F0aW9uPjxwdWJsaXNoZXI+VW5pdmVyc2l0eSBvZiBIYXdhaSZhcG9zO2ksIFNlY29uZCBM
YW5ndWFnZSBUZWFjaGluZyBhbmQgQ3VycmljdWx1bSBDZW50ZXI8L3B1Ymxpc2hlcj48aXNibj4x
NzwvaXNibj48Y2FsbC1udW0+UDUzIFc2NSAxOTk4PC9jYWxsLW51bT48d29yay10eXBlPlRlY2hu
aWNhbCBSZXBvcnQ8L3dvcmstdHlwZT48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5k
Tm90ZT5=
ADDIN EN.CITE.DATA (a pattern observed in many L2 writing studies with intermediate-level writers; see, for example, Ortega, 2003; Wolfe-Quintero et al., 1998) before relying more on phrasal elaboration ADDIN EN.CITE <EndNote><Cite><Author>Biber</Author><Year>2006</Year><RecNum>395</RecNum><Prefix>a pattern observed by </Prefix><Suffix> with advanced L1 academic writers</Suffix><DisplayText>(a pattern observed by Biber, 2006 with advanced L1 academic writers)</DisplayText><record><rec-number>395</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">395</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Biber, Douglas</author></authors></contributors><titles><title>University language: A corpus-based study of spoken and written registers</title><secondary-title>Studies in corpus linguistics, 1388-0373 ; v. 23</secondary-title></titles><number>viii, 261 p. :</number><keywords><keyword>Academic language Data processing.</keyword><keyword>Register (Linguistics) Data processing.</keyword></keywords><dates><year>2006</year></dates><pub-location>Amsterdam</pub-location><publisher>John Benjamins</publisher><isbn>9027222959
9789027222954
9027222967 (pbk.)
9789027222961 (pbk.)</isbn><urls><related-urls><url>Table of contents only ;(a pattern observed by Biber, 2006 with advanced L1 academic writers). Considering the level of the students in the current study—high-intermediate to advanced L2 and not advanced L1 writers—one would expect to see more expansion through subordination rather than at the phrase level. PEVuZE5vdGU+PENpdGUgRXhjbHVkZUF1dGg9IjEiIEV4Y2x1ZGVZZWFyPSIxIiBIaWRkZW49IjEi
PjxBdXRob3I+Tm9ycmlzPC9BdXRob3I+PFllYXI+MjAwOTwvWWVhcj48UmVjTnVtPjM2NjwvUmVj
TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MzY2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl
eSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4z
NjY8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x
NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPk5vcnJpcywgSm9obiBN
LjwvYXV0aG9yPjxhdXRob3I+T3J0ZWdhLCBMb3VyZGVzPC9hdXRob3I+PC9hdXRob3JzPjwvY29u
dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlRvd2FyZHMgYW4gb3JnYW5pYyBhcHByb2FjaCB0byBp
bnZlc3RpZ2F0aW5nIENBRiBpbiBpbnN0cnVjdGVkIFNMQTogVGhlIGNhc2Ugb2YgY29tcGxleGl0
eTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5BcHBsaWVkIExpbmd1aXN0aWNzPC9zZWNvbmRhcnkt
dGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QXBwbGllZCBMaW5ndWlzdGlj
czwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjU1NS01Nzg8L3BhZ2VzPjx2b2x1bWU+
MzA8L3ZvbHVtZT48bnVtYmVyPjQ8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48cHVi
LWRhdGVzPjxkYXRlPkRlY2VtYmVyIDEsIDIwMDk8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48
dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2FwcGxpai5veGZvcmRqb3VybmFscy5vcmcv
Y29udGVudC8zMC80LzU1NS5mdWxsPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJv
bmljLXJlc291cmNlLW51bT4xMC4xMDkzL2FwcGxpbi9hbXAwNDQ8L2VsZWN0cm9uaWMtcmVzb3Vy
Y2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPk5vcnJpczwvQXV0aG9yPjxZZWFy
PjIwMDk8L1llYXI+PFJlY051bT4zNjY8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM2Njwv
cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndl
ZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+MzY2PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10
eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1
dGhvcnM+PGF1dGhvcj5Ob3JyaXMsIEpvaG4gTS48L2F1dGhvcj48YXV0aG9yPk9ydGVnYSwgTG91
cmRlczwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Ub3dh
cmRzIGFuIG9yZ2FuaWMgYXBwcm9hY2ggdG8gaW52ZXN0aWdhdGluZyBDQUYgaW4gaW5zdHJ1Y3Rl
ZCBTTEE6IFRoZSBjYXNlIG9mIGNvbXBsZXhpdHk8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBw
bGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxm
dWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxw
YWdlcz41NTUtNTc4PC9wYWdlcz48dm9sdW1lPjMwPC92b2x1bWU+PG51bWJlcj40PC9udW1iZXI+
PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5EZWNlbWJlciAxLCAyMDA5
PC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6
Ly9hcHBsaWoub3hmb3Jkam91cm5hbHMub3JnL2NvbnRlbnQvMzAvNC81NTUuZnVsbDwvdXJsPjwv
cmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA5My9hcHBs
aW4vYW1wMDQ0PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGUg
QXV0aG9yWWVhcj0iMSI+PEF1dGhvcj5Ob3JyaXM8L0F1dGhvcj48WWVhcj4yMDA5PC9ZZWFyPjxS
ZWNOdW0+MzY2PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zNjY8L3JlYy1udW1iZXI+PGZv
cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVz
cGF4eHhyZnpmc3ciPjM2Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3Vy
bmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+
Tm9ycmlzLCBKb2huIE0uPC9hdXRob3I+PGF1dGhvcj5PcnRlZ2EsIExvdXJkZXM8L2F1dGhvcj48
L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VG93YXJkcyBhbiBvcmdhbmlj
IGFwcHJvYWNoIHRvIGludmVzdGlnYXRpbmcgQ0FGIGluIGluc3RydWN0ZWQgU0xBOiBUaGUgY2Fz
ZSBvZiBjb21wbGV4aXR5PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3Rp
Y3M8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5BcHBs
aWVkIExpbmd1aXN0aWNzPC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NTU1LTU3ODwv
cGFnZXM+PHZvbHVtZT4zMDwvdm9sdW1lPjxudW1iZXI+NDwvbnVtYmVyPjxkYXRlcz48eWVhcj4y
MDA5PC95ZWFyPjxwdWItZGF0ZXM+PGRhdGU+RGVjZW1iZXIgMSwgMjAwOTwvZGF0ZT48L3B1Yi1k
YXRlcz48L2RhdGVzPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vYXBwbGlqLm94Zm9y
ZGpvdXJuYWxzLm9yZy9jb250ZW50LzMwLzQvNTU1LmZ1bGw8L3VybD48L3JlbGF0ZWQtdXJscz48
L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwOTMvYXBwbGluL2FtcDA0NDwvZWxl
Y3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT4A
ADDIN EN.CITE PEVuZE5vdGU+PENpdGUgRXhjbHVkZUF1dGg9IjEiIEV4Y2x1ZGVZZWFyPSIxIiBIaWRkZW49IjEi
PjxBdXRob3I+Tm9ycmlzPC9BdXRob3I+PFllYXI+MjAwOTwvWWVhcj48UmVjTnVtPjM2NjwvUmVj
TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MzY2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl
eSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4z
NjY8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x
NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPk5vcnJpcywgSm9obiBN
LjwvYXV0aG9yPjxhdXRob3I+T3J0ZWdhLCBMb3VyZGVzPC9hdXRob3I+PC9hdXRob3JzPjwvY29u
dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlRvd2FyZHMgYW4gb3JnYW5pYyBhcHByb2FjaCB0byBp
bnZlc3RpZ2F0aW5nIENBRiBpbiBpbnN0cnVjdGVkIFNMQTogVGhlIGNhc2Ugb2YgY29tcGxleGl0
eTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5BcHBsaWVkIExpbmd1aXN0aWNzPC9zZWNvbmRhcnkt
dGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QXBwbGllZCBMaW5ndWlzdGlj
czwvZnVsbC10aXRsZT48L3BlcmlvZGljYWw+PHBhZ2VzPjU1NS01Nzg8L3BhZ2VzPjx2b2x1bWU+
MzA8L3ZvbHVtZT48bnVtYmVyPjQ8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48cHVi
LWRhdGVzPjxkYXRlPkRlY2VtYmVyIDEsIDIwMDk8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48
dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2FwcGxpai5veGZvcmRqb3VybmFscy5vcmcv
Y29udGVudC8zMC80LzU1NS5mdWxsPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJv
bmljLXJlc291cmNlLW51bT4xMC4xMDkzL2FwcGxpbi9hbXAwNDQ8L2VsZWN0cm9uaWMtcmVzb3Vy
Y2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPk5vcnJpczwvQXV0aG9yPjxZZWFy
PjIwMDk8L1llYXI+PFJlY051bT4zNjY8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM2Njwv
cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndl
ZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+MzY2PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10
eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1
dGhvcnM+PGF1dGhvcj5Ob3JyaXMsIEpvaG4gTS48L2F1dGhvcj48YXV0aG9yPk9ydGVnYSwgTG91
cmRlczwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Ub3dh
cmRzIGFuIG9yZ2FuaWMgYXBwcm9hY2ggdG8gaW52ZXN0aWdhdGluZyBDQUYgaW4gaW5zdHJ1Y3Rl
ZCBTTEE6IFRoZSBjYXNlIG9mIGNvbXBsZXhpdHk8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QXBw
bGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxm
dWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2FsPjxw
YWdlcz41NTUtNTc4PC9wYWdlcz48dm9sdW1lPjMwPC92b2x1bWU+PG51bWJlcj40PC9udW1iZXI+
PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5EZWNlbWJlciAxLCAyMDA5
PC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6
Ly9hcHBsaWoub3hmb3Jkam91cm5hbHMub3JnL2NvbnRlbnQvMzAvNC81NTUuZnVsbDwvdXJsPjwv
cmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTA5My9hcHBs
aW4vYW1wMDQ0PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGUg
QXV0aG9yWWVhcj0iMSI+PEF1dGhvcj5Ob3JyaXM8L0F1dGhvcj48WWVhcj4yMDA5PC9ZZWFyPjxS
ZWNOdW0+MzY2PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zNjY8L3JlYy1udW1iZXI+PGZv
cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVz
cGF4eHhyZnpmc3ciPjM2Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3Vy
bmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+
Tm9ycmlzLCBKb2huIE0uPC9hdXRob3I+PGF1dGhvcj5PcnRlZ2EsIExvdXJkZXM8L2F1dGhvcj48
L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VG93YXJkcyBhbiBvcmdhbmlj
IGFwcHJvYWNoIHRvIGludmVzdGlnYXRpbmcgQ0FGIGluIGluc3RydWN0ZWQgU0xBOiBUaGUgY2Fz
ZSBvZiBjb21wbGV4aXR5PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3Rp
Y3M8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5BcHBs
aWVkIExpbmd1aXN0aWNzPC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NTU1LTU3ODwv
cGFnZXM+PHZvbHVtZT4zMDwvdm9sdW1lPjxudW1iZXI+NDwvbnVtYmVyPjxkYXRlcz48eWVhcj4y
MDA5PC95ZWFyPjxwdWItZGF0ZXM+PGRhdGU+RGVjZW1iZXIgMSwgMjAwOTwvZGF0ZT48L3B1Yi1k
YXRlcz48L2RhdGVzPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vYXBwbGlqLm94Zm9y
ZGpvdXJuYWxzLm9yZy9jb250ZW50LzMwLzQvNTU1LmZ1bGw8L3VybD48L3JlbGF0ZWQtdXJscz48
L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwOTMvYXBwbGluL2FtcDA0NDwvZWxl
Y3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT4A
ADDIN EN.CITE.DATA Production units and morphological and syntactic errors were also hand-tagged blindly by a second coder with experience in teaching and rating L2 writing. After a training session, the second coder worked independently to code and tally clauses, sentences, T-units, and errors for 50% of the exams in each course. Interrater reliability was analyzed by calculating the absolute agreement for intraclass correlations with a two-way mixed model using SPSS. The interclass coefficients for single measures ranged from 0.96 for clauses to 0.88 for errors. Following Polio ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Polio</Author><Year>1997</Year><RecNum>68</RecNum><DisplayText>(1997)</DisplayText><record><rec-number>68</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">68</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">583</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Polio, Charlene G.</author></authors></contributors><titles><title>Measures of linguistic accuracy in second language writing research</title><secondary-title>Language Learning</secondary-title></titles><periodical><full-title>Language Learning</full-title></periodical><pages>101-143</pages><volume>47</volume><number>1</number><dates><year>1997</year></dates><urls></urls><electronic-resource-num>10.1111/0023-8333.31997003</electronic-resource-num></record></Cite></EndNote>(1997), discrepancies were resolved by averaging the tallies from both coders.To examine the relationships between the accuracy and complexity measures and the teacher assigned grammar grade, Pearson product-moment correlation coefficients between the teacher-assigned grammar grade and the accuracy and complexity measures were calculated for each course. The grammar grade used for this analysis is the subscore assigned by the teacher on the grammar criterion on the program evaluation grid (see Appendix A). It therefore contributes to the overall score on the essay but is otherwise disarticulated from that overall grade.Phase 2 and 3: Questionnaire and interviews with students and teachers (QUAL) (data to answer research questions 1, 2, and 3).I used open and axial coding techniques associated with developing grounded theory to identify themes and categories in the questionnaire and interview data ADDIN EN.CITE <EndNote><Cite><Author>Corbin</Author><Year>2008</Year><RecNum>261</RecNum><DisplayText>(Corbin & Strauss, 2008)</DisplayText><record><rec-number>261</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">261</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">763</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Corbin, Juliet M.</author><author>Strauss, Anselm L.</author></authors><secondary-authors><author>Strauss, Anselm L.</author></secondary-authors></contributors><titles><title>Basics of qualitative research: Techniques and procedures for developing grounded theory</title></titles><pages>xv, 379 p. :</pages><edition>3rd</edition><keywords><keyword>Social sciences -- Statistical methods</keyword><keyword>Grounded theory</keyword></keywords><dates><year>2008</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>9781412906449 (pbk. : alk. paper)
9781412906432 (cloth : alk. paper)</isbn><call-num>HA 29 S823 2008</call-num><urls></urls></record></Cite></EndNote>(Corbin & Strauss, 2008). Open coding requires the analysis of the data without drawing on preconceived codes or categories. Rather, the codes and concepts emerge from careful reading of, reflecting on, and thinking about the data and continue to develop as the process of analysis continues. The use and appearance of each coding category in the data is continuously cross-referenced with other instances to clarify and define each code and concept more clearly using the raw data ADDIN EN.CITE <EndNote><Cite><Author>Corbin</Author><Year>2008</Year><RecNum>261</RecNum><DisplayText>(Corbin & Strauss, 2008)</DisplayText><record><rec-number>261</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">261</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">763</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Corbin, Juliet M.</author><author>Strauss, Anselm L.</author></authors><secondary-authors><author>Strauss, Anselm L.</author></secondary-authors></contributors><titles><title>Basics of qualitative research: Techniques and procedures for developing grounded theory</title></titles><pages>xv, 379 p. :</pages><edition>3rd</edition><keywords><keyword>Social sciences -- Statistical methods</keyword><keyword>Grounded theory</keyword></keywords><dates><year>2008</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>9781412906449 (pbk. : alk. paper)
9781412906432 (cloth : alk. paper)</isbn><call-num>HA 29 S823 2008</call-num><urls></urls></record></Cite></EndNote>(Corbin & Strauss, 2008) and therefore keeps the researcher close to the raw data and not his/her preconceived notions ADDIN EN.CITE <EndNote><Cite><Author>Charmaz</Author><Year>2003</Year><RecNum>314</RecNum><DisplayText>(Charmaz, 2003)</DisplayText><record><rec-number>314</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">314</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">814</key></foreign-keys><ref-type name="Book Section">5</ref-type><contributors><authors><author>Charmaz, Kathy</author></authors><secondary-authors><author>Holstein, James A.</author><author>Gubrium, Jaber F.</author></secondary-authors></contributors><titles><title>Qualitative interviewing and grounded theory analysis</title><secondary-title>Inside interviewing: New lenses, new concerns</secondary-title></titles><pages>311-330</pages><dates><year>2003</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><urls></urls></record></Cite></EndNote>(Charmaz, 2003). In addition, I kept memos to record the reasoning and analysis behind the emerging concepts and categories PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Db3JiaW48L0F1dGhvcj48WWVhcj4yMDA4PC9ZZWFyPjxS
ZWNOdW0+MjYxPC9SZWNOdW0+PERpc3BsYXlUZXh0PihDaGFybWF6LCAyMDAzLCAyMDA2OyBDb3Ji
aW4gJmFtcDsgU3RyYXVzcywgMjAwOCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+
MjYxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBk
OTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yNjE8L2tleT48a2V5IGFwcD0iRU5XZWIi
IGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43NjM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVm
LXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkNvcmJpbiwgSnVsaWV0IE0uPC9hdXRob3I+PGF1dGhvcj5TdHJhdXNzLCBBbnNlbG0gTC48
L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1hdXRob3JzPjxhdXRob3I+U3RyYXVzcywgQW5z
ZWxtIEwuPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5CYXNpY3Mgb2YgcXVhbGl0YXRpdmUgcmVzZWFyY2g6IFRlY2huaXF1ZXMgYW5kIHBy
b2NlZHVyZXMgZm9yIGRldmVsb3BpbmcgZ3JvdW5kZWQgdGhlb3J5PC90aXRsZT48L3RpdGxlcz48
cGFnZXM+eHYsIDM3OSBwLiA6PC9wYWdlcz48ZWRpdGlvbj4zcmQ8L2VkaXRpb24+PGtleXdvcmRz
PjxrZXl3b3JkPlNvY2lhbCBzY2llbmNlcyAtLSBTdGF0aXN0aWNhbCBtZXRob2RzPC9rZXl3b3Jk
PjxrZXl3b3JkPkdyb3VuZGVkIHRoZW9yeTwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVh
cj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5UaG91c2FuZCBPYWtzLCBDQTwvcHVi
LWxvY2F0aW9uPjxwdWJsaXNoZXI+U0FHRTwvcHVibGlzaGVyPjxpc2JuPjk3ODE0MTI5MDY0NDkg
KHBiay4gOiBhbGsuIHBhcGVyKSYjeEQ7OTc4MTQxMjkwNjQzMiAoY2xvdGggOiBhbGsuIHBhcGVy
KTwvaXNibj48Y2FsbC1udW0+SEEgMjkgUzgyMyAyMDA4PC9jYWxsLW51bT48dXJscz48L3VybHM+
PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Q2hhcm1hejwvQXV0aG9yPjxZZWFyPjIwMDY8
L1llYXI+PFJlY051bT4zMTA8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjMxMDwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2
MHBwOTlydXNwYXh4eHJmemZzdyI+MzEwPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dS
RGV3cnRxZ2NBQUFwdk1DRSI+ODEwPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5DaGFybWF6
LCBLYXRoeTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5D
b25zdHJ1Y3RpbmcgZ3JvdW5kZWQgdGhlb3J5OiBBIHByYWN0aWNhbCBndWlkZSB0aHJvdWdoIHF1
YWxpdGF0aXZlIGFuYWx5c2lzPC90aXRsZT48L3RpdGxlcz48bnVtYmVyPnhpaWksIDIwOCBwLiA6
PC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPkdyb3VuZGVkIHRoZW9yeS48L2tleXdvcmQ+PGtl
eXdvcmQ+U29jaWFsIHNjaWVuY2VzIFJlc2VhcmNoIE1ldGhvZG9sb2d5Ljwva2V5d29yZD48L2tl
eXdvcmRzPjxkYXRlcz48eWVhcj4yMDA2PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5UaG91
c2FuZCBPYWtzLCBDQTwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+U0FHRTwvcHVibGlzaGVyPjxp
c2JuPjA3NjE5NzM1MjQgKGhhcmRjb3ZlcikmI3hEOzk3ODA3NjE5NzM1MjIgKGhhcmRjb3Zlcikm
I3hEOzA3NjE5NzM1MzIgKHBiay4pJiN4RDs5NzgwNzYxOTczNTM5IChwYmsuKSYjeEQ7OTc4MDc2
MTk3MzUzOSYjeEQ7OTc4MDc2MTk3MzUyMjwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+
UHVibGlzaGVyIGRlc2NyaXB0aW9uIGh0dHA6Ly93d3cubG9jLmdvdi9jYXRkaXIvZW5oYW5jZW1l
bnRzL2Z5MDY1Ny8yMDA1OTI4MDM1LWQuaHRtbDwvdXJsPjx1cmw+VGFibGUgb2YgY29udGVudHMg
aHR0cDovL3d3dy5sb2MuZ292L2NhdGRpci9lbmhhbmNlbWVudHMvZnkwNjU3LzIwMDU5MjgwMzUt
dC5odG1sPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48
QXV0aG9yPkNoYXJtYXo8L0F1dGhvcj48WWVhcj4yMDAzPC9ZZWFyPjxSZWNOdW0+MzE0PC9SZWNO
dW0+PHJlY29yZD48cmVjLW51bWJlcj4zMTQ8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjMx
NDwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjgxNDwv
a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIFNlY3Rpb24iPjU8L3JlZi10
eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5DaGFybWF6LCBLYXRoeTwvYXV0aG9y
PjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1dGhvcnM+PGF1dGhvcj5Ib2xzdGVpbiwgSmFtZXMgQS48
L2F1dGhvcj48YXV0aG9yPkd1YnJpdW0sIEphYmVyIEYuPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0
aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5RdWFsaXRhdGl2ZSBpbnRlcnZpZXdp
bmcgYW5kIGdyb3VuZGVkIHRoZW9yeSBhbmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5J
bnNpZGUgaW50ZXJ2aWV3aW5nOiBOZXcgbGVuc2VzLCBuZXcgY29uY2VybnM8L3NlY29uZGFyeS10
aXRsZT48L3RpdGxlcz48cGFnZXM+MzExLTMzMDwvcGFnZXM+PGRhdGVzPjx5ZWFyPjIwMDM8L3ll
YXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPlRob3VzYW5kIE9ha3MsIENBPC9wdWItbG9jYXRpb24+
PHB1Ymxpc2hlcj5TQUdFPC9wdWJsaXNoZXI+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48
L0VuZE5vdGU+
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Db3JiaW48L0F1dGhvcj48WWVhcj4yMDA4PC9ZZWFyPjxS
ZWNOdW0+MjYxPC9SZWNOdW0+PERpc3BsYXlUZXh0PihDaGFybWF6LCAyMDAzLCAyMDA2OyBDb3Ji
aW4gJmFtcDsgU3RyYXVzcywgMjAwOCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+
MjYxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBk
OTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4yNjE8L2tleT48a2V5IGFwcD0iRU5XZWIi
IGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj43NjM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVm
LXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkNvcmJpbiwgSnVsaWV0IE0uPC9hdXRob3I+PGF1dGhvcj5TdHJhdXNzLCBBbnNlbG0gTC48
L2F1dGhvcj48L2F1dGhvcnM+PHNlY29uZGFyeS1hdXRob3JzPjxhdXRob3I+U3RyYXVzcywgQW5z
ZWxtIEwuPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5CYXNpY3Mgb2YgcXVhbGl0YXRpdmUgcmVzZWFyY2g6IFRlY2huaXF1ZXMgYW5kIHBy
b2NlZHVyZXMgZm9yIGRldmVsb3BpbmcgZ3JvdW5kZWQgdGhlb3J5PC90aXRsZT48L3RpdGxlcz48
cGFnZXM+eHYsIDM3OSBwLiA6PC9wYWdlcz48ZWRpdGlvbj4zcmQ8L2VkaXRpb24+PGtleXdvcmRz
PjxrZXl3b3JkPlNvY2lhbCBzY2llbmNlcyAtLSBTdGF0aXN0aWNhbCBtZXRob2RzPC9rZXl3b3Jk
PjxrZXl3b3JkPkdyb3VuZGVkIHRoZW9yeTwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVh
cj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5UaG91c2FuZCBPYWtzLCBDQTwvcHVi
LWxvY2F0aW9uPjxwdWJsaXNoZXI+U0FHRTwvcHVibGlzaGVyPjxpc2JuPjk3ODE0MTI5MDY0NDkg
KHBiay4gOiBhbGsuIHBhcGVyKSYjeEQ7OTc4MTQxMjkwNjQzMiAoY2xvdGggOiBhbGsuIHBhcGVy
KTwvaXNibj48Y2FsbC1udW0+SEEgMjkgUzgyMyAyMDA4PC9jYWxsLW51bT48dXJscz48L3VybHM+
PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Q2hhcm1hejwvQXV0aG9yPjxZZWFyPjIwMDY8
L1llYXI+PFJlY051bT4zMTA8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjMxMDwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2
MHBwOTlydXNwYXh4eHJmemZzdyI+MzEwPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dS
RGV3cnRxZ2NBQUFwdk1DRSI+ODEwPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkJvb2siPjY8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5DaGFybWF6
LCBLYXRoeTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5D
b25zdHJ1Y3RpbmcgZ3JvdW5kZWQgdGhlb3J5OiBBIHByYWN0aWNhbCBndWlkZSB0aHJvdWdoIHF1
YWxpdGF0aXZlIGFuYWx5c2lzPC90aXRsZT48L3RpdGxlcz48bnVtYmVyPnhpaWksIDIwOCBwLiA6
PC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPkdyb3VuZGVkIHRoZW9yeS48L2tleXdvcmQ+PGtl
eXdvcmQ+U29jaWFsIHNjaWVuY2VzIFJlc2VhcmNoIE1ldGhvZG9sb2d5Ljwva2V5d29yZD48L2tl
eXdvcmRzPjxkYXRlcz48eWVhcj4yMDA2PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5UaG91
c2FuZCBPYWtzLCBDQTwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+U0FHRTwvcHVibGlzaGVyPjxp
c2JuPjA3NjE5NzM1MjQgKGhhcmRjb3ZlcikmI3hEOzk3ODA3NjE5NzM1MjIgKGhhcmRjb3Zlcikm
I3hEOzA3NjE5NzM1MzIgKHBiay4pJiN4RDs5NzgwNzYxOTczNTM5IChwYmsuKSYjeEQ7OTc4MDc2
MTk3MzUzOSYjeEQ7OTc4MDc2MTk3MzUyMjwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+
UHVibGlzaGVyIGRlc2NyaXB0aW9uIGh0dHA6Ly93d3cubG9jLmdvdi9jYXRkaXIvZW5oYW5jZW1l
bnRzL2Z5MDY1Ny8yMDA1OTI4MDM1LWQuaHRtbDwvdXJsPjx1cmw+VGFibGUgb2YgY29udGVudHMg
aHR0cDovL3d3dy5sb2MuZ292L2NhdGRpci9lbmhhbmNlbWVudHMvZnkwNjU3LzIwMDU5MjgwMzUt
dC5odG1sPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48
QXV0aG9yPkNoYXJtYXo8L0F1dGhvcj48WWVhcj4yMDAzPC9ZZWFyPjxSZWNOdW0+MzE0PC9SZWNO
dW0+PHJlY29yZD48cmVjLW51bWJlcj4zMTQ8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjMx
NDwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRld3J0cWdjQUFBcHZNQ0UiPjgxNDwv
a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIFNlY3Rpb24iPjU8L3JlZi10
eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5DaGFybWF6LCBLYXRoeTwvYXV0aG9y
PjwvYXV0aG9ycz48c2Vjb25kYXJ5LWF1dGhvcnM+PGF1dGhvcj5Ib2xzdGVpbiwgSmFtZXMgQS48
L2F1dGhvcj48YXV0aG9yPkd1YnJpdW0sIEphYmVyIEYuPC9hdXRob3I+PC9zZWNvbmRhcnktYXV0
aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5RdWFsaXRhdGl2ZSBpbnRlcnZpZXdp
bmcgYW5kIGdyb3VuZGVkIHRoZW9yeSBhbmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5J
bnNpZGUgaW50ZXJ2aWV3aW5nOiBOZXcgbGVuc2VzLCBuZXcgY29uY2VybnM8L3NlY29uZGFyeS10
aXRsZT48L3RpdGxlcz48cGFnZXM+MzExLTMzMDwvcGFnZXM+PGRhdGVzPjx5ZWFyPjIwMDM8L3ll
YXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPlRob3VzYW5kIE9ha3MsIENBPC9wdWItbG9jYXRpb24+
PHB1Ymxpc2hlcj5TQUdFPC9wdWJsaXNoZXI+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48
L0VuZE5vdGU+
ADDIN EN.CITE.DATA (Charmaz, 2003, 2006; Corbin & Strauss, 2008). Furthermore, these memos helped me to clarify the concepts and categories I noticed in the data as the analysis of the transcripts progressed. At the same time, I used axial coding; this means that I related the emerging concepts to each other to understand the relationships among them ADDIN EN.CITE <EndNote><Cite><Author>Corbin</Author><Year>2008</Year><RecNum>261</RecNum><DisplayText>(Corbin & Strauss, 2008)</DisplayText><record><rec-number>261</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">261</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">763</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Corbin, Juliet M.</author><author>Strauss, Anselm L.</author></authors><secondary-authors><author>Strauss, Anselm L.</author></secondary-authors></contributors><titles><title>Basics of qualitative research: Techniques and procedures for developing grounded theory</title></titles><pages>xv, 379 p. :</pages><edition>3rd</edition><keywords><keyword>Social sciences -- Statistical methods</keyword><keyword>Grounded theory</keyword></keywords><dates><year>2008</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>9781412906449 (pbk. : alk. paper)
9781412906432 (cloth : alk. paper)</isbn><call-num>HA 29 S823 2008</call-num><urls></urls></record></Cite></EndNote>(Corbin & Strauss, 2008). Using this process, information on the teachers’ assessment criteria and process as well as problems, concerns, or issues expressed by both teachers and students emerged. In subsequent analyses and readings of these interviews, the concepts and categories were refined through more detailed cross-referencing between different participants’ accounts and the clarification of concepts and themes. Two methods were used to validate these analyses: the triangulation technique and peer debriefing PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5DcmVzd2VsbDwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+
PFJlY051bT4zNDc8L1JlY051bT48RGlzcGxheVRleHQ+KENyZXN3ZWxsLCAyMDA5OyBDcmVzd2Vs
bCAmYW1wOyBNaWxsZXIsIDIwMDA7IExpbmNvbG4gJmFtcDsgR3ViYSwgMTk4NTsgVGVkZGxpZSAm
YW1wOyBUYXNoYWtrb3JpLCAyMDA5KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4z
NDc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5
NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjM0Nzwva2V5PjwvZm9yZWlnbi1rZXlzPjxy
ZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+Q3Jlc3dlbGwsIEpvaG4gVy48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+UmVzZWFyY2ggZGVzaWduOiBRdWFsaXRhdGl2ZSwgcXVhbnRpdGF0aXZl
LCBhbmQgbWl4ZWQgbWV0aG9kcyBhcHByb2FjaGVzPC90aXRsZT48L3RpdGxlcz48bnVtYmVyPnh4
aXgsIDI2MCBwLiA7PC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPlNvY2lhbCBzY2llbmNlcyBS
ZXNlYXJjaCBNZXRob2RvbG9neS48L2tleXdvcmQ+PGtleXdvcmQ+U29jaWFsIHNjaWVuY2VzIFN0
YXRpc3RpY2FsIG1ldGhvZHMuPC9rZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMDk8
L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPlRob3VzYW5kIE9ha3MsIENBPC9wdWItbG9jYXRp
b24+PHB1Ymxpc2hlcj5TQUdFPC9wdWJsaXNoZXI+PGlzYm4+OTc4MTQxMjk2NTU2OSAoY2xvdGgp
JiN4RDsxNDEyOTY1NTZYIChjbG90aCkmI3hEOzk3ODE0MTI5NjU1NzYgKHBiay4pJiN4RDsxNDEy
OTY1NTc4IChwYmsuKTwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxB
dXRob3I+Q3Jlc3dlbGw8L0F1dGhvcj48WWVhcj4yMDAwPC9ZZWFyPjxSZWNOdW0+MzQzPC9SZWNO
dW0+PHJlY29yZD48cmVjLW51bWJlcj4zNDM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjM0
Mzwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3
PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Q3Jlc3dlbGwsIEpvaG4g
Vy48L2F1dGhvcj48YXV0aG9yPk1pbGxlciwgRGFuYSBMLjwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5EZXRlcm1pbmluZyB2YWxpZGl0eSBpbiBxdWFsaXRh
dGl2ZSBpbnF1aXJ5PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlRoZW9yeSBpbnRvIFByYWN0aWNl
PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjEyNC0xMzA8L3BhZ2VzPjx2b2x1bWU+
Mzk8L3ZvbHVtZT48bnVtYmVyPjM8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwMDwveWVhcj48L2Rh
dGVzPjxwdWJsaXNoZXI+VGF5bG9yICZhbXA7IEZyYW5jaXMsIEx0ZC48L3B1Ymxpc2hlcj48aXNi
bj4wMDQwNTg0MTwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL3d3dy5qc3Rv
ci5vcmcvc3RhYmxlLzE0Nzc1NDM8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9u
aWMtcmVzb3VyY2UtbnVtPjEwLjEyMDcvczE1NDMwNDIxdGlwMzkwM18yPC9lbGVjdHJvbmljLXJl
c291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5MaW5jb2xuPC9BdXRob3I+
PFllYXI+MTk4NTwvWWVhcj48UmVjTnVtPjM0MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+
MzQyPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBk
OTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4zNDI8L2tleT48L2ZvcmVpZ24ta2V5cz48
cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48
YXV0aG9yPkxpbmNvbG4sIFl2b25uYSBTLjwvYXV0aG9yPjxhdXRob3I+R3ViYSwgRWdvbiBHLjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5OYXR1cmFsaXN0
aWMgaW5xdWlyeTwvdGl0bGU+PC90aXRsZXM+PGtleXdvcmRzPjxrZXl3b3JkPk5hdHVyYWxpc20u
PC9rZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjE5ODU8L3llYXI+PC9kYXRlcz48cHVi
LWxvY2F0aW9uPkJldmVybHkgSGlsbHMsIENBPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5TQUdF
PC9wdWJsaXNoZXI+PGlzYm4+MDgwMzkyNDMxMzwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+
PC9DaXRlPjxDaXRlPjxBdXRob3I+VGVkZGxpZTwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJl
Y051bT4yOTI8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjI5MjwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNw
YXh4eHJmemZzdyI+MjkyPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NB
QUFwdk1DRSI+NzkyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8
L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5UZWRkbGllLCBDaGFybGVz
PC9hdXRob3I+PGF1dGhvcj5UYXNoYWtrb3JpLCBBYmJhczwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Gb3VuZGF0aW9ucyBvZiBtaXhlZCBtZXRob2RzIHJl
c2VhcmNoOiBJbnRlZ3JhdGluZyBxdWFudGl0YXRpdmUgYW5kIHF1YWxpdGF0aXZlIGFwcHJvYWNo
ZXMgaW4gdGhlIHNvY2lhbCBhbmQgYmVoYXZpb3JhbCBzY2llbmNlczwvdGl0bGU+PC90aXRsZXM+
PGtleXdvcmRzPjxrZXl3b3JkPlNvY2lhbCBzY2llbmNlcyBSZXNlYXJjaCBNZXRob2RvbG9neS48
L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48L2RhdGVzPjxwdWIt
bG9jYXRpb24+VGhvdXNhbmQgT2FrcywgQ0E8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlNBR0U8
L3B1Ymxpc2hlcj48aXNibj45NzgwNzYxOTMwMTEyIChjbG90aCkmI3hEOzA3NjE5MzAxMTYgKGNs
b3RoKSYjeEQ7OTc4MDc2MTkzMDEyOSAocGJrLikmI3hEOzA3NjE5MzAxMjQgKHBiay4pPC9pc2Ju
Pjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5DcmVzd2VsbDwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+
PFJlY051bT4zNDc8L1JlY051bT48RGlzcGxheVRleHQ+KENyZXN3ZWxsLCAyMDA5OyBDcmVzd2Vs
bCAmYW1wOyBNaWxsZXIsIDIwMDA7IExpbmNvbG4gJmFtcDsgR3ViYSwgMTk4NTsgVGVkZGxpZSAm
YW1wOyBUYXNoYWtrb3JpLCAyMDA5KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4z
NDc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5
NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjM0Nzwva2V5PjwvZm9yZWlnbi1rZXlzPjxy
ZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+Q3Jlc3dlbGwsIEpvaG4gVy48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+UmVzZWFyY2ggZGVzaWduOiBRdWFsaXRhdGl2ZSwgcXVhbnRpdGF0aXZl
LCBhbmQgbWl4ZWQgbWV0aG9kcyBhcHByb2FjaGVzPC90aXRsZT48L3RpdGxlcz48bnVtYmVyPnh4
aXgsIDI2MCBwLiA7PC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPlNvY2lhbCBzY2llbmNlcyBS
ZXNlYXJjaCBNZXRob2RvbG9neS48L2tleXdvcmQ+PGtleXdvcmQ+U29jaWFsIHNjaWVuY2VzIFN0
YXRpc3RpY2FsIG1ldGhvZHMuPC9rZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMDk8
L3llYXI+PC9kYXRlcz48cHViLWxvY2F0aW9uPlRob3VzYW5kIE9ha3MsIENBPC9wdWItbG9jYXRp
b24+PHB1Ymxpc2hlcj5TQUdFPC9wdWJsaXNoZXI+PGlzYm4+OTc4MTQxMjk2NTU2OSAoY2xvdGgp
JiN4RDsxNDEyOTY1NTZYIChjbG90aCkmI3hEOzk3ODE0MTI5NjU1NzYgKHBiay4pJiN4RDsxNDEy
OTY1NTc4IChwYmsuKTwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxB
dXRob3I+Q3Jlc3dlbGw8L0F1dGhvcj48WWVhcj4yMDAwPC9ZZWFyPjxSZWNOdW0+MzQzPC9SZWNO
dW0+PHJlY29yZD48cmVjLW51bWJlcj4zNDM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpmc3ciPjM0
Mzwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3
PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Q3Jlc3dlbGwsIEpvaG4g
Vy48L2F1dGhvcj48YXV0aG9yPk1pbGxlciwgRGFuYSBMLjwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5EZXRlcm1pbmluZyB2YWxpZGl0eSBpbiBxdWFsaXRh
dGl2ZSBpbnF1aXJ5PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlRoZW9yeSBpbnRvIFByYWN0aWNl
PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjEyNC0xMzA8L3BhZ2VzPjx2b2x1bWU+
Mzk8L3ZvbHVtZT48bnVtYmVyPjM8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwMDwveWVhcj48L2Rh
dGVzPjxwdWJsaXNoZXI+VGF5bG9yICZhbXA7IEZyYW5jaXMsIEx0ZC48L3B1Ymxpc2hlcj48aXNi
bj4wMDQwNTg0MTwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL3d3dy5qc3Rv
ci5vcmcvc3RhYmxlLzE0Nzc1NDM8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9u
aWMtcmVzb3VyY2UtbnVtPjEwLjEyMDcvczE1NDMwNDIxdGlwMzkwM18yPC9lbGVjdHJvbmljLXJl
c291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5MaW5jb2xuPC9BdXRob3I+
PFllYXI+MTk4NTwvWWVhcj48UmVjTnVtPjM0MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+
MzQyPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBk
OTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3Ij4zNDI8L2tleT48L2ZvcmVpZ24ta2V5cz48
cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48
YXV0aG9yPkxpbmNvbG4sIFl2b25uYSBTLjwvYXV0aG9yPjxhdXRob3I+R3ViYSwgRWdvbiBHLjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5OYXR1cmFsaXN0
aWMgaW5xdWlyeTwvdGl0bGU+PC90aXRsZXM+PGtleXdvcmRzPjxrZXl3b3JkPk5hdHVyYWxpc20u
PC9rZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjE5ODU8L3llYXI+PC9kYXRlcz48cHVi
LWxvY2F0aW9uPkJldmVybHkgSGlsbHMsIENBPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5TQUdF
PC9wdWJsaXNoZXI+PGlzYm4+MDgwMzkyNDMxMzwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+
PC9DaXRlPjxDaXRlPjxBdXRob3I+VGVkZGxpZTwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJl
Y051bT4yOTI8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjI5MjwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNw
YXh4eHJmemZzdyI+MjkyPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NB
QUFwdk1DRSI+NzkyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8
L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5UZWRkbGllLCBDaGFybGVz
PC9hdXRob3I+PGF1dGhvcj5UYXNoYWtrb3JpLCBBYmJhczwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Gb3VuZGF0aW9ucyBvZiBtaXhlZCBtZXRob2RzIHJl
c2VhcmNoOiBJbnRlZ3JhdGluZyBxdWFudGl0YXRpdmUgYW5kIHF1YWxpdGF0aXZlIGFwcHJvYWNo
ZXMgaW4gdGhlIHNvY2lhbCBhbmQgYmVoYXZpb3JhbCBzY2llbmNlczwvdGl0bGU+PC90aXRsZXM+
PGtleXdvcmRzPjxrZXl3b3JkPlNvY2lhbCBzY2llbmNlcyBSZXNlYXJjaCBNZXRob2RvbG9neS48
L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48L2RhdGVzPjxwdWIt
bG9jYXRpb24+VGhvdXNhbmQgT2FrcywgQ0E8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlNBR0U8
L3B1Ymxpc2hlcj48aXNibj45NzgwNzYxOTMwMTEyIChjbG90aCkmI3hEOzA3NjE5MzAxMTYgKGNs
b3RoKSYjeEQ7OTc4MDc2MTkzMDEyOSAocGJrLikmI3hEOzA3NjE5MzAxMjQgKHBiay4pPC9pc2Ju
Pjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE.DATA (Creswell, 2009; Creswell & Miller, 2000; Lincoln & Guba, 1985; Teddlie & Tashakkori, 2009). For the student interviews, information from different participants was used to triangulate findings. The analysis of the student and teacher interviews was also conducted by a second researcher with a PhD in Language Assessment. She was familiar with the research project and reviewed the data, the emerging codes and themes, and the presentation of findings to assess the validity of the analysis. ResultsPhase 1: Text-based Analysis (QUAN)Table 2 presents the descriptive statistics for the teacher-assigned grammar grade and the results for the accuracy and complexity measures for each course. Whereas the results in the two courses are similar for the two complexity measures, the figures for the accuracy measure and the grammar grade differ between the two courses. The mean and standard deviation for the grammar grade is higher in Course 2, but for the errors per 100 words ratio these values are lower for this course. In both courses, the variables have similar ranges with a certain degree of variance in relation to the grammar grade with a somewhat wider range for the errors per 100 words ratio in Course 1 and the words per clause ratio in Course 2. One outlier in Course 2 was removed from the data set.[Insert Table 2 about here]Correlation coefficients were computed between the teacher-assigned grammar grade and the accuracy and complexity measures. The results of the correlational analysis for each course presented in Table 3 show that only the strong negative correlation of -.848 between the grammar grade and the errors per 100 words ratio in Course 2 is statistically significant. The other correlations are moderate to weak and not statistically significant. Interestingly, there appears to be a moderate negative correlation between clause per T-unit ratio and the teacher-assigned grammar grade, albeit not statistically significant, whereas we would expect to see a positive correlation between these two variables.[Insert Table 3 about here]Phase 2: Student Questionnaires and Interviews (QUAL)Thirty-three students completed the questionnaire, and eight students were also interviewed. The presentation of the findings focuses on the two most important themes that emerged from the analysis: the role of accuracy and complexity in the assessment of grammar and the impact of the teachers’ assessment criteria on students’ writing and learning. A more detailed discussion of additional themes can be found in Neumann ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Neumann</Author><Year>2011</Year><RecNum>446</RecNum><DisplayText>(2011)</DisplayText><record><rec-number>446</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">446</key></foreign-keys><ref-type name="Thesis">32</ref-type><contributors><authors><author>Neumann, Heike</author></authors></contributors><titles><title>What's in a grade? A mixed methods investigation of teacher assessment of grammatical ability in L2 academic writing</title><secondary-title>McGill theses</secondary-title></titles><dates><year>2011</year></dates><pub-location>Montreal</pub-location><publisher>McGill University</publisher><work-type>Unpublished doctoral dissertation</work-type><urls><related-urls><url>;(2011). The findings are organized by theme rather than by data source, and, where possible, findings from both data sources have been included for each theme. For certain themes, however, only data from the interviews were available, in which case this is explicitly mentioned below.Grammatical accuracy, sentence variety, and complexity.Grammatical accuracy emerged as an important theme in the questionnaire and interview data. About 85% of the student participants in Course 1 refer directly or indirectly to accuracy as the teacher’s assessment criterion for grammar. Some of these students simply list items from the evaluation grid that teachers can tick or circle as part of their evaluation, such as “sentence problems” or “verb structures” (see Appendix B). Since teachers use these lists during the assessment process to identify strong and weak elements, one can deduce that students believe their teachers assess the level of accuracy on the aspects of morphology and/or syntax that students mentioned on the questionnaire. Almost 75% of the student participants in Course 2 mention accuracy or grammar mistakes as important indicators of grammatical ability for the teacher. A small number of student participants in both courses also mention broader issues in relation to grammar assessment, as seen in the following statements: “That she will find stuff that will prove our understanding of her lessons.” (Pierre, Course 1); “she tries to help us improving. After that, she’ll know which grammar points she needs to talk about in class” (Emilie, Course 2); or “I think the teacher look to improve our skills in English and help us learn from those mistakes.” (Gabriela, Course 2). These statements show the students’ awareness of the relationship between the teachers’ instructional goals and their assessment procedures and criteria. Accuracy also emerged as a prevalent theme in student responses to the question of whether their teacher’s expectations affect the way they write their essays. Ten students in both courses indicated on the student questionnaire and/or in the interview that they try to avoid making grammatical mistakes because they know this is what the teacher looks for. The following statement from Xing’s (Course 2) interview is one example: “But I think the most important thing is grammar. If your grammar is perfect, maybe you can get an A- or B+”. According to two of the eight student participants who were interviewed, sentence variety and complexity is another important assessment criterion for teachers. The teachers want students “to make [their] sentence[s] look nice” (Yi, Course 2) through the use of conjunctions and transition phrases. For this purpose, teachers provide students with support in order to enable them to use conjunctions and transition words. The goal of producing precise and varied sentences that capture the complexity of the writer’s ideas is not easily achieved, however, because one of the main concerns for students is to avoid errors. This means that these students have to negotiate and choose between “safe” linguistic options over which they have full control and “risky” options that would allow them to express more complex ideas and illustrate relationships between these ideas. Emilie explained this dilemma during the interview:In fact what really happens if I really, if this idea is very important and I really want it to be in my essay because sometimes I have complex idea and I don’t really know how to write them? I just forget them and I say something easier unfortunately because of the grammar. But sometimes when I know that the rest of the essay is quite well written and there is no mistakes in it, I say ok I can make one sentence with a lot of mistakes in it I don’t mind. ... But it is like once or twice for an essay.On essay exams, however, Emilie would not take this risk. Instead, she “will put the idea in simple sentences … [and] split [the idea] in various sentences”. Like Emilie, four other students report that they deliberately opt for simple sentence constructions and avoid compound, complex, and compound-complex sentences, all in an effort to circumvent making grammatical mistakes. These students resort to familiar sentence structures in order to avoid making mistakes with unfamiliar or new constructions, as the following quotes illustrate: “So in the exam I think the classmate is like me and we always write the things we are very familiar with to diminish the mistakes” (Xing) and “I try to use structure I am comfortable with to avoid mistakes” (Gabriela). Eight students also referred to this idea in their questionnaire responses. At the same time, these students hope that when they do select the more challenging and complex option that they will be rewarded for this by the teacher, as suggested by Emilie: “Because I hope the teacher ... will appreciate the fact that I am making an effort to write a complex sentence and maybe she will.” Emilie is uncertain, however, whether the teacher will appreciate her linguistic efforts and her risk-taking behaviour.Assessment and learning.During the student interviews, it emerged that the grade assigned as a result of the assessment process has a negative influence on the willingness of some students in this study to take risks, and, ultimately, this may impact their learning. How and whether the grade affects the students depends on the relative importance of the grade in relation to the course mark. As illustrated in Emilie’s quote in the preceding section, the grammar grade sometimes keeps her from trying out new language, especially if the grade is important. She feels that she needs space to experiment with language without the fear of losing points for grammar mistakes. This is echoed by Xing’s comment: “Maybe just it is obviously a spelling mistake but it is always wrong. So the grades always discourage me”. However, if students feel they have the space to take risks without “being punished” for grammar mistakes and without a numeric assessment that counts towards the course grade, they feel that they have a better opportunity to learn as Xing points out: [It is different] when we write … the sentence practice because not contain any marks in my final grade. So I can write what I want to write and maybe I am not so sure about this sentence but I can write it in the paper and let teacher to correct it. Yeah, so I think it is the good way for me to … improve my grammar.The exchange with the teacher and her feedback are obviously very important for the students’ learning, but marking or grades may also have a negative effect. It seems that the students need a space without marks to really improve and expand their language repertoire because under exam conditions and in other assessment situations they may stick to the familiar and not take risks. According to the students of Course 1, both first and final drafts where graded, and there was little ungraded writing practice, so these students would have not had this opportunity. In Course 2, however, students completed ungraded sentence writing assignments with the target vocabulary for the course, and first drafts of essay assignments were ungraded.Assessment of grammar: A summary of the students’ perspective.Students in this study agree with the teachers’ focus on grammatical accuracy as an assessment criterion for grammar. At the same time, this focus has unintended consequences. For a substantial number of these students, there is a trade-off between producing grammatically accurate language and the ability to use more complex sentence structures. Because these students know that grammatical errors lead to lower grammar grades, they avoid challenging language structures and stick to those structures that they know well. This avoidance behaviour affects their learning negatively because students have the impression that risk-taking behaviour is not rewarded, or at least they are not certain whether it is.Phase 3: Teacher Interviews (QUAL)Assessment criteria for grammar.Grammatical accuracy also emerged as an important theme in the teacher interviews. Miriam, the teacher of Course 1, uses accuracy as the main criterion to assess her students’ grammatical ability in writing and has developed her own method of assigning scores on the evaluation grid: Typically what I do when I am going through an essay I actually tick errors that they are making … I’ll tally it all up and I look at it that way. … So if they have twenty tallies on [the evaluation grid] then I am just going to subtract from forty and that is how I will get their mark.Her rationale for using this system is two-fold. First, it makes the assessment of grammar very clear and transparent to the students; they know exactly why they obtained a certain grade. Second, this system visually identifies and clearly communicates areas of weakness to the students: If students have more than five errors in one category, Miriam assigns extra work. As was evident in the student interviews, some students may opt to avoid making errors by choosing safer and simpler sentence constructions over which they have good control. According to Miriam, however, these students would not gain any advantage by doing that in her course because sentence variety and evidence of complex constructions are also criteria for her; she refers to the band descriptors that accompany the evaluation grid to ensure that students receive the appropriate grade that is reflective of the level of accuracy as well as complexity in their writing. During the discussion of the marking criteria for selected essays in her class, Miriam pointed particularly to the error count as a criterion for determining the grammar grade on the evaluation grid.Elaine, the teacher of Course 2, looks for what she calls “tolerably acceptable language.” She gives low or failing marks “if the back of my head goes screech.” The biggest criterion for this intuitive reaction to her students’ writing is comprehensibility. In the discussion of the four essay exams, it became apparent that “acceptable language” also had to be accurate because Elaine pointed out different kinds of language errors as she was reviewing the essays during the interview. Like Miriam, Elaine marks errors on the evaluation grid, but she does not use a mathematical formula to determine the grammar grade. Rather, she looks not only at the level of accuracy but also for variety of expression and generally “elegant language” as reflected in vocabulary use, sentence structure, and the evident thought process. That is, Elaine considers the three constructs present in the evaluation grid descriptors: comprehensibility of the text, effectiveness of sentence construction, and accuracy (see Appendix A). For the low grades, comprehensibility and clarity of the ideas are always an issue. During the discussion of the marking criteria for particular essays in her class, Elaine makes reference to a number of criteria. While accuracy and the overall count of grammatical errors do have an impact, she also refers to concepts such as “workable language” or “variety of expression” when explaining why she assigned a particular grammar grade for essays. Therefore, in addition to accuracy, she assesses the ease with which she can understand the ideas expressed in the text (“workable language”) as well as the repertoire of linguistic expression that is evident in the student’s essay (“variety of expression”). The focus of grammatical accuracy as an instructional goal is also reflected in what Elaine refers to as her weekly sentence writing practice, where students have to submit individual sentences and focus on writing them as grammatically accurately as possible.Assessment of grammar: A summary of the teachers’ perspective.Both teachers attend to accuracy in the students’ writing although this criterion plays a different role for each teacher in the assessment process. For Miriam, it seems that practical concerns have led her to adopt her method of tallying errors to determine the grammar grade. Elaine, on the other hand, relies on her intuitive judgement when she looks for “acceptable language.” This criterion means that the message should be comprehensible and that communication should not be impeded by linguistic errors. Consequently, Elaine also focuses on linguistic accuracy. The teachers’ focus on accuracy is motivated by their desire to encourage their students to improve their command of English grammar and their control of linguistic structures. That is, the teachers hope that the focus on accuracy will lead to student learning: Students will notice their mistakes and learn to avoid them in the future. The unintended consequence, however, is that students may avoid certain structures altogether in an effort to reduce grammatical errors. These avoidance strategies may ultimately have a negative impact on student learning, as the findings from the student interviews indicate.Discussion & ImplicationsAnswers to Research Questions (RQ)RQ 1: What indicators of grammatical ability do teachers attend to in their students’ L2 academic writing when assigning grades for grammar on an analytical rating scale?The results from this study point to accuracy as an important indicator of grammatical ability for teachers in this study. Although the correlation between the grammar grade and the accuracy measure for Course 1 was not statistically significant, the teacher Miriam repeatedly mentioned accuracy as an assessment criterion during the interview, and she bases her assessment of grammatical ability on the overall error count for each essay. In contrast, the accuracy measure shows a strong negative correlation with the grammar grade in Course 2 despite the teacher’s comment that she was not focused on an overall error count. The lack of statistical significance of the correlation between the accuracy measure and the grammar grade for Course 1 may be at least partly due to the smaller variance of grammar scores as evident in the smaller standard deviation of 5.53 compared to that of 10.41 for Course 2 ADDIN EN.CITE <EndNote><Cite><Author>Ortega</Author><Year>2003</Year><RecNum>66</RecNum><Prefix>see Table 2`; cf. </Prefix><DisplayText>(see Table 2; cf. Ortega, 2003)</DisplayText><record><rec-number>66</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">66</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">581</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Ortega, Lourdes</author></authors></contributors><titles><title>Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>492-518</pages><volume>24</volume><number>4</number><dates><year>2003</year></dates><urls><related-urls><url> </electronic-resource-num></record></Cite></EndNote>(see Table 2; cf. Ortega, 2003). Although sentence variety and complexity are important criteria for higher grammar grades for Elaine according to the interview data, the negative correlation (albeit not statistically significant) between the clauses per T-unit ratio and the grammar grade points towards the possibility that the opposite may be the case: Complexity may in fact lead to lower grammar grades. There is only a weak positive trend observed in that course for the words per clause ratio. It could be argued that the descriptive statistics for the two courses potentially indicate different stages of development in terms of complexification of the L2 writers’ texts with the writers in Course 2 somewhat more advanced as apparent in the slightly lower mean for the clauses per T-unit ratio and slightly higher mean for the words per clause ratio in Course 2. This difference, however, is not evident in any differences in how the teachers describe their process of assigning the grammar grade or in different statistically significant correlations between these measures and the grammar grade for the two courses. Overall, the data point to sentence-level indicators of accuracy as the primary assessment criterion. RQ 2: What knowledge do students have about the indicators their teachers look for when assigning a grade for grammar in writing?The students in this study are aware of their teachers’ focus on accuracy. They can enumerate particular areas of weaknesses that their teachers had previously identified in their writing using the evaluation grid.RQ 3: What do students perceive to be the impact of their teachers’ assessment criteria on their way of writing and learning in the L2 writing classroom?The students in this study apparently attempt to understand their teachers’ expectations and then tailor their writing to meet those expectations. Rather than focusing on their learning and taking risks to experiment with new language and expand their linguistic repertoire, students may avoid particular structures in order to satisfy the teachers’ expectation for accuracy.Accuracy, Complexity, and the Teacher-Assigned Grammar GradeBoth the text analysis and the interview data indicate that grammatical accuracy is the main indicator for the teacher-assigned grammar grade. The errors per 100 words ratio is negatively correlated to the teacher-assigned grammar grade in Course 2. In other words, as the number of errors decreases and the level of accuracy increases, the grammar grade goes up. This finding was to be expected because rating scales for L2 writing tasks in high-stakes tests such as the TOEFL ADDIN EN.CITE <EndNote><Cite><Author>Educational Testing Service</Author><Year>2004</Year><RecNum>178</RecNum><DisplayText>(Educational Testing Service, 2004)</DisplayText><record><rec-number>178</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">178</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">685</key></foreign-keys><ref-type name="Web Page">12</ref-type><contributors><authors><author>Educational Testing Service,</author></authors></contributors><titles><title>Scoring Guides (Rubrics) for Writing Responses.</title></titles><number>April 8, 2007</number><dates><year>2004</year></dates><urls><related-urls><url> under "iBT TOEFL Scoring Guides"</url></related-urls></urls></record></Cite></EndNote>(Educational Testing Service, 2004) or the IELTS ADDIN EN.CITE <EndNote><Cite><Author>International English Language Testing System</Author><Year>n.d.</Year><RecNum>252</RecNum><DisplayText>(International English Language Testing System, n.d.-a, n.d.-b)</DisplayText><record><rec-number>252</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">252</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">754</key></foreign-keys><ref-type name="Web Page">12</ref-type><contributors><authors><author>International English Language Testing System,</author></authors></contributors><titles><title>IELTS Task 1 writing band descriptors.</title></titles><number>June 10, 2009</number><dates><year>n.d.</year></dates><urls><related-urls><url> English Language Testing System</Author><Year>n.d.</Year><RecNum>251</RecNum><record><rec-number>251</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">251</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">753</key></foreign-keys><ref-type name="Web Page">12</ref-type><contributors><authors><author>International English Language Testing System,</author></authors></contributors><titles><title>IELTS Task 2 writing band descriptors.</title></titles><number>June 10, 2009</number><dates><year>n.d.</year></dates><urls><related-urls><url>;(International English Language Testing System, n.d.-a, n.d.-b) also include increasing demands for accuracy with every band increase on the rating scales. Just like the raters on those tests, the teachers in this study look for increased levels of accuracy in order to award higher grammar grades on the evaluation grid. Because of the emphasis on grammatical accuracy in class, in feedback, and on the program evaluation grids, students are aware of this evaluation criterion for grammar.In contrast, there is no statistically significant correlation between the two complexity measures and the grammar grade. Interestingly, for one of the measures we can notice a trend towards a negative correlation between the clauses per T-unit ratio and the grammar grade in Course 2. This is surprising because increased complexity is usually considered a sign of increased proficiency and more advanced language use and control PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Ob3JyaXM8L0F1dGhvcj48WWVhcj4yMDA5PC9ZZWFyPjxS
ZWNOdW0+MzY2PC9SZWNOdW0+PERpc3BsYXlUZXh0PihOb3JyaXMgJmFtcDsgT3J0ZWdhLCAyMDA5
OyBPcnRlZ2EsIDIwMDM7IFNrZWhhbiwgMjAwOTsgV29sZmUtUXVpbnRlcm8gZXQgYWwuLCAxOTk4
KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4zNjY8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4
eHhyZnpmc3ciPjM2Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFs
IEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Tm9y
cmlzLCBKb2huIE0uPC9hdXRob3I+PGF1dGhvcj5PcnRlZ2EsIExvdXJkZXM8L2F1dGhvcj48L2F1
dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VG93YXJkcyBhbiBvcmdhbmljIGFw
cHJvYWNoIHRvIGludmVzdGlnYXRpbmcgQ0FGIGluIGluc3RydWN0ZWQgU0xBOiBUaGUgY2FzZSBv
ZiBjb21wbGV4aXR5PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8
L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5BcHBsaWVk
IExpbmd1aXN0aWNzPC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NTU1LTU3ODwvcGFn
ZXM+PHZvbHVtZT4zMDwvdm9sdW1lPjxudW1iZXI+NDwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDA5
PC95ZWFyPjxwdWItZGF0ZXM+PGRhdGU+RGVjZW1iZXIgMSwgMjAwOTwvZGF0ZT48L3B1Yi1kYXRl
cz48L2RhdGVzPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vYXBwbGlqLm94Zm9yZGpv
dXJuYWxzLm9yZy9jb250ZW50LzMwLzQvNTU1LmZ1bGw8L3VybD48L3JlbGF0ZWQtdXJscz48L3Vy
bHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwOTMvYXBwbGluL2FtcDA0NDwvZWxlY3Ry
b25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+T3J0ZWdhPC9B
dXRob3I+PFllYXI+MjAwMzwvWWVhcj48UmVjTnVtPjY2PC9SZWNOdW0+PHJlY29yZD48cmVjLW51
bWJlcj42NjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4
cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NjY8L2tleT48a2V5IGFwcD0iRU5X
ZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41ODE8L2tleT48L2ZvcmVpZ24ta2V5cz48
cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y
cz48YXV0aG9ycz48YXV0aG9yPk9ydGVnYSwgTG91cmRlczwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5TeW50YWN0aWMgY29tcGxleGl0eSBtZWFzdXJlcyBh
bmQgdGhlaXIgcmVsYXRpb25zaGlwIHRvIEwyIHByb2ZpY2llbmN5OiBBIHJlc2VhcmNoIHN5bnRo
ZXNpcyBvZiBjb2xsZWdlLWxldmVsIEwyIHdyaXRpbmc8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+
QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2Fs
PjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2Fs
PjxwYWdlcz40OTItNTE4PC9wYWdlcz48dm9sdW1lPjI0PC92b2x1bWU+PG51bWJlcj40PC9udW1i
ZXI+PGRhdGVzPjx5ZWFyPjIwMDM8L3llYXI+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1
cmw+aHR0cDovL2FwcGxpai5veGZvcmRqb3VybmFscy5vcmcvY29udGVudC8yNC80LzQ5Mi5mdWxs
LnBkZitodG1sPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNl
LW51bT4xMC4xMDkzL2FwcGxpbi8yNC40LjQ5MiA8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwv
cmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNrZWhhbjwvQXV0aG9yPjxZZWFyPjIwMDk8L1ll
YXI+PFJlY051bT4zNjM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM2MzwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBw
OTlydXNwYXh4eHJmemZzdyI+MzYzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5Ta2VoYW4sIFBldGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRs
ZXM+PHRpdGxlPk1vZGVsbGluZyBzZWNvbmQgbGFuZ3VhZ2UgcGVyZm9ybWFuY2U6IEludGVncmF0
aW5nIGNvbXBsZXhpdHksIGFjY3VyYWN5LCBmbHVlbmN5LCBhbmQgbGV4aXM8L3RpdGxlPjxzZWNv
bmRhcnktdGl0bGU+QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVz
PjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+
PC9wZXJpb2RpY2FsPjxwYWdlcz41MTAtNTMyPC9wYWdlcz48dm9sdW1lPjMwPC92b2x1bWU+PG51
bWJlcj40PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5E
ZWNlbWJlciAxLCAyMDA5PC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQt
dXJscz48dXJsPmh0dHA6Ly9hcHBsaWoub3hmb3Jkam91cm5hbHMub3JnL2NnaS9jb250ZW50L2Fi
c3RyYWN0LzMwLzQvNTEwPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJl
c291cmNlLW51bT4xMC4xMDkzL2FwcGxpbi9hbXAwNDc8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVt
PjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPldvbGZlLVF1aW50ZXJvPC9BdXRob3I+PFll
YXI+MTk5ODwvWWVhcj48UmVjTnVtPjEzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4xMzwv
cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndl
ZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+MTM8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlk
PSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41Mzg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUg
bmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldv
bGZlLVF1aW50ZXJvLCBLYXRlPC9hdXRob3I+PGF1dGhvcj5JbmFnYWtpLCBTaHVuamk8L2F1dGhv
cj48YXV0aG9yPktpbSwgSGFlLVlvdW5nPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPlNlY29uZCBsYW5ndWFnZSBkZXZlbG9wbWVudCBpbiB3cml0aW5nOiBN
ZWFzdXJlcyBvZiBmbHVlbmN5LCBhY2N1cmFjeSAmYW1wOyBjb21wbGV4aXR5PC90aXRsZT48L3Rp
dGxlcz48cGFnZXM+MTg3PC9wYWdlcz48ZGF0ZXM+PHllYXI+MTk5ODwveWVhcj48L2RhdGVzPjxw
dWItbG9jYXRpb24+SG9ub2x1bHUsIEhJPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5Vbml2ZXJz
aXR5IG9mIEhhd2FpJmFwb3M7aSwgU2Vjb25kIExhbmd1YWdlIFRlYWNoaW5nIGFuZCBDdXJyaWN1
bHVtIENlbnRlcjwvcHVibGlzaGVyPjxpc2JuPjE3PC9pc2JuPjxjYWxsLW51bT5QNTMgVzY1IDE5
OTg8L2NhbGwtbnVtPjx3b3JrLXR5cGU+VGVjaG5pY2FsIFJlcG9ydDwvd29yay10eXBlPjx1cmxz
PjwvdXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Ob3JyaXM8L0F1dGhvcj48WWVhcj4yMDA5PC9ZZWFyPjxS
ZWNOdW0+MzY2PC9SZWNOdW0+PERpc3BsYXlUZXh0PihOb3JyaXMgJmFtcDsgT3J0ZWdhLCAyMDA5
OyBPcnRlZ2EsIDIwMDM7IFNrZWhhbiwgMjAwOTsgV29sZmUtUXVpbnRlcm8gZXQgYWwuLCAxOTk4
KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4zNjY8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4
eHhyZnpmc3ciPjM2Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFs
IEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Tm9y
cmlzLCBKb2huIE0uPC9hdXRob3I+PGF1dGhvcj5PcnRlZ2EsIExvdXJkZXM8L2F1dGhvcj48L2F1
dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VG93YXJkcyBhbiBvcmdhbmljIGFw
cHJvYWNoIHRvIGludmVzdGlnYXRpbmcgQ0FGIGluIGluc3RydWN0ZWQgU0xBOiBUaGUgY2FzZSBv
ZiBjb21wbGV4aXR5PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8
L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5BcHBsaWVk
IExpbmd1aXN0aWNzPC9mdWxsLXRpdGxlPjwvcGVyaW9kaWNhbD48cGFnZXM+NTU1LTU3ODwvcGFn
ZXM+PHZvbHVtZT4zMDwvdm9sdW1lPjxudW1iZXI+NDwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDA5
PC95ZWFyPjxwdWItZGF0ZXM+PGRhdGU+RGVjZW1iZXIgMSwgMjAwOTwvZGF0ZT48L3B1Yi1kYXRl
cz48L2RhdGVzPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vYXBwbGlqLm94Zm9yZGpv
dXJuYWxzLm9yZy9jb250ZW50LzMwLzQvNTU1LmZ1bGw8L3VybD48L3JlbGF0ZWQtdXJscz48L3Vy
bHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwOTMvYXBwbGluL2FtcDA0NDwvZWxlY3Ry
b25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+T3J0ZWdhPC9B
dXRob3I+PFllYXI+MjAwMzwvWWVhcj48UmVjTnVtPjY2PC9SZWNOdW0+PHJlY29yZD48cmVjLW51
bWJlcj42NjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4
cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+NjY8L2tleT48a2V5IGFwcD0iRU5X
ZWIiIGRiLWlkPSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41ODE8L2tleT48L2ZvcmVpZ24ta2V5cz48
cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y
cz48YXV0aG9ycz48YXV0aG9yPk9ydGVnYSwgTG91cmRlczwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5TeW50YWN0aWMgY29tcGxleGl0eSBtZWFzdXJlcyBh
bmQgdGhlaXIgcmVsYXRpb25zaGlwIHRvIEwyIHByb2ZpY2llbmN5OiBBIHJlc2VhcmNoIHN5bnRo
ZXNpcyBvZiBjb2xsZWdlLWxldmVsIEwyIHdyaXRpbmc8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+
QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2Fs
PjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+PC9wZXJpb2RpY2Fs
PjxwYWdlcz40OTItNTE4PC9wYWdlcz48dm9sdW1lPjI0PC92b2x1bWU+PG51bWJlcj40PC9udW1i
ZXI+PGRhdGVzPjx5ZWFyPjIwMDM8L3llYXI+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1
cmw+aHR0cDovL2FwcGxpai5veGZvcmRqb3VybmFscy5vcmcvY29udGVudC8yNC80LzQ5Mi5mdWxs
LnBkZitodG1sPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNl
LW51bT4xMC4xMDkzL2FwcGxpbi8yNC40LjQ5MiA8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwv
cmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNrZWhhbjwvQXV0aG9yPjxZZWFyPjIwMDk8L1ll
YXI+PFJlY051bT4zNjM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM2MzwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBw
OTlydXNwYXh4eHJmemZzdyI+MzYzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9
IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5Ta2VoYW4sIFBldGVyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRs
ZXM+PHRpdGxlPk1vZGVsbGluZyBzZWNvbmQgbGFuZ3VhZ2UgcGVyZm9ybWFuY2U6IEludGVncmF0
aW5nIGNvbXBsZXhpdHksIGFjY3VyYWN5LCBmbHVlbmN5LCBhbmQgbGV4aXM8L3RpdGxlPjxzZWNv
bmRhcnktdGl0bGU+QXBwbGllZCBMaW5ndWlzdGljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVz
PjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFwcGxpZWQgTGluZ3Vpc3RpY3M8L2Z1bGwtdGl0bGU+
PC9wZXJpb2RpY2FsPjxwYWdlcz41MTAtNTMyPC9wYWdlcz48dm9sdW1lPjMwPC92b2x1bWU+PG51
bWJlcj40PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5E
ZWNlbWJlciAxLCAyMDA5PC9kYXRlPjwvcHViLWRhdGVzPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQt
dXJscz48dXJsPmh0dHA6Ly9hcHBsaWoub3hmb3Jkam91cm5hbHMub3JnL2NnaS9jb250ZW50L2Fi
c3RyYWN0LzMwLzQvNTEwPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJl
c291cmNlLW51bT4xMC4xMDkzL2FwcGxpbi9hbXAwNDc8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVt
PjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPldvbGZlLVF1aW50ZXJvPC9BdXRob3I+PFll
YXI+MTk5ODwvWWVhcj48UmVjTnVtPjEzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4xMzwv
cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndl
ZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+MTM8L2tleT48a2V5IGFwcD0iRU5XZWIiIGRiLWlk
PSJTZ1JEZXdydHFnY0FBQXB2TUNFIj41Mzg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUg
bmFtZT0iQm9vayI+NjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldv
bGZlLVF1aW50ZXJvLCBLYXRlPC9hdXRob3I+PGF1dGhvcj5JbmFnYWtpLCBTaHVuamk8L2F1dGhv
cj48YXV0aG9yPktpbSwgSGFlLVlvdW5nPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPlNlY29uZCBsYW5ndWFnZSBkZXZlbG9wbWVudCBpbiB3cml0aW5nOiBN
ZWFzdXJlcyBvZiBmbHVlbmN5LCBhY2N1cmFjeSAmYW1wOyBjb21wbGV4aXR5PC90aXRsZT48L3Rp
dGxlcz48cGFnZXM+MTg3PC9wYWdlcz48ZGF0ZXM+PHllYXI+MTk5ODwveWVhcj48L2RhdGVzPjxw
dWItbG9jYXRpb24+SG9ub2x1bHUsIEhJPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5Vbml2ZXJz
aXR5IG9mIEhhd2FpJmFwb3M7aSwgU2Vjb25kIExhbmd1YWdlIFRlYWNoaW5nIGFuZCBDdXJyaWN1
bHVtIENlbnRlcjwvcHVibGlzaGVyPjxpc2JuPjE3PC9pc2JuPjxjYWxsLW51bT5QNTMgVzY1IDE5
OTg8L2NhbGwtbnVtPjx3b3JrLXR5cGU+VGVjaG5pY2FsIFJlcG9ydDwvd29yay10eXBlPjx1cmxz
PjwvdXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Norris & Ortega, 2009; Ortega, 2003; Skehan, 2009; Wolfe-Quintero et al., 1998). Consequently, when L2 writers produce more complex sentences, the teacher-assigned grammar grade should increase and not decrease, as appears to be the case in Course 2. This finding would also be in conflict with the descriptors of the program evaluation grid, where the top band describes the language as varied and complex (see Appendix B). Language use at the low bands, on the other hand, is characterized by frequent problems in or avoidance of complex structures (see Appendix B). This expectation for increased complexity at higher bands of a rating scale is also reflected in rating scales for high-stakes exams PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5JbnRlcm5hdGlvbmFsIEVuZ2xpc2ggTGFuZ3VhZ2UgVGVz
dGluZyBTeXN0ZW08L0F1dGhvcj48WWVhcj5uLmQuPC9ZZWFyPjxSZWNOdW0+MjUyPC9SZWNOdW0+
PERpc3BsYXlUZXh0PihFZHVjYXRpb25hbCBUZXN0aW5nIFNlcnZpY2UsIDIwMDQ7IEludGVybmF0
aW9uYWwgRW5nbGlzaCBMYW5ndWFnZSBUZXN0aW5nIFN5c3RlbSwgbi5kLi1hLCBuLmQuLWIpPC9E
aXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjI1MjwvcmVjLW51bWJlcj48Zm9yZWlnbi1r
ZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJm
emZzdyI+MjUyPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1D
RSI+NzU0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IldlYiBQYWdlIj4xMjwv
cmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkludGVybmF0aW9uYWwgRW5n
bGlzaCBMYW5ndWFnZSBUZXN0aW5nIFN5c3RlbSw8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmli
dXRvcnM+PHRpdGxlcz48dGl0bGU+SUVMVFMgVGFzayAxIHdyaXRpbmcgYmFuZCBkZXNjcmlwdG9y
cy48L3RpdGxlPjwvdGl0bGVzPjxudW1iZXI+SnVuZSAxMCwgMjAwOTwvbnVtYmVyPjxkYXRlcz48
eWVhcj5uLmQuPC95ZWFyPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93
d3cuaWVsdHMub3JnL3Jlc2VhcmNoZXJzL3Njb3JlX3Byb2Nlc3NpbmdfYW5kX3JlcG9ydGluZy5h
c3B4PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0
aG9yPkVkdWNhdGlvbmFsIFRlc3RpbmcgU2VydmljZTwvQXV0aG9yPjxZZWFyPjIwMDQ8L1llYXI+
PFJlY051bT4xNzg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjE3ODwvcmVjLW51bWJlcj48
Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTly
dXNwYXh4eHJmemZzdyI+MTc4PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRx
Z2NBQUFwdk1DRSI+Njg1PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IldlYiBQ
YWdlIj4xMjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkVkdWNhdGlv
bmFsIFRlc3RpbmcgU2VydmljZSw8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRp
dGxlcz48dGl0bGU+U2NvcmluZyBHdWlkZXMgKFJ1YnJpY3MpIGZvciBXcml0aW5nIFJlc3BvbnNl
cy48L3RpdGxlPjwvdGl0bGVzPjxudW1iZXI+QXByaWwgOCwgMjAwNzwvbnVtYmVyPjxkYXRlcz48
eWVhcj4yMDA0PC95ZWFyPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPnd3dy5ldHMu
b3JnIHVuZGVyICZxdW90O2lCVCBUT0VGTCBTY29yaW5nIEd1aWRlcyZxdW90OzwvdXJsPjwvcmVs
YXRlZC11cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5JbnRlcm5hdGlv
bmFsIEVuZ2xpc2ggTGFuZ3VhZ2UgVGVzdGluZyBTeXN0ZW08L0F1dGhvcj48WWVhcj5uLmQuPC9Z
ZWFyPjxSZWNOdW0+MjUxPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNTE8L3JlYy1udW1i
ZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBw
cDk5cnVzcGF4eHhyZnpmc3ciPjI1MTwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRl
d3J0cWdjQUFBcHZNQ0UiPjc1Mzwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJX
ZWIgUGFnZSI+MTI8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5JbnRl
cm5hdGlvbmFsIEVuZ2xpc2ggTGFuZ3VhZ2UgVGVzdGluZyBTeXN0ZW0sPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPklFTFRTIFRhc2sgMiB3cml0aW5nIGJh
bmQgZGVzY3JpcHRvcnMuPC90aXRsZT48L3RpdGxlcz48bnVtYmVyPkp1bmUgMTAsIDIwMDk8L251
bWJlcj48ZGF0ZXM+PHllYXI+bi5kLjwveWVhcj48L2RhdGVzPjx1cmxzPjxyZWxhdGVkLXVybHM+
PHVybD5odHRwOi8vd3d3LmllbHRzLm9yZy9yZXNlYXJjaGVycy9zY29yZV9wcm9jZXNzaW5nX2Fu
ZF9yZXBvcnRpbmcuYXNweDwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48L3JlY29yZD48L0Np
dGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5JbnRlcm5hdGlvbmFsIEVuZ2xpc2ggTGFuZ3VhZ2UgVGVz
dGluZyBTeXN0ZW08L0F1dGhvcj48WWVhcj5uLmQuPC9ZZWFyPjxSZWNOdW0+MjUyPC9SZWNOdW0+
PERpc3BsYXlUZXh0PihFZHVjYXRpb25hbCBUZXN0aW5nIFNlcnZpY2UsIDIwMDQ7IEludGVybmF0
aW9uYWwgRW5nbGlzaCBMYW5ndWFnZSBUZXN0aW5nIFN5c3RlbSwgbi5kLi1hLCBuLmQuLWIpPC9E
aXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjI1MjwvcmVjLW51bWJlcj48Zm9yZWlnbi1r
ZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJm
emZzdyI+MjUyPC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRxZ2NBQUFwdk1D
RSI+NzU0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IldlYiBQYWdlIj4xMjwv
cmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkludGVybmF0aW9uYWwgRW5n
bGlzaCBMYW5ndWFnZSBUZXN0aW5nIFN5c3RlbSw8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmli
dXRvcnM+PHRpdGxlcz48dGl0bGU+SUVMVFMgVGFzayAxIHdyaXRpbmcgYmFuZCBkZXNjcmlwdG9y
cy48L3RpdGxlPjwvdGl0bGVzPjxudW1iZXI+SnVuZSAxMCwgMjAwOTwvbnVtYmVyPjxkYXRlcz48
eWVhcj5uLmQuPC95ZWFyPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93
d3cuaWVsdHMub3JnL3Jlc2VhcmNoZXJzL3Njb3JlX3Byb2Nlc3NpbmdfYW5kX3JlcG9ydGluZy5h
c3B4PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0
aG9yPkVkdWNhdGlvbmFsIFRlc3RpbmcgU2VydmljZTwvQXV0aG9yPjxZZWFyPjIwMDQ8L1llYXI+
PFJlY051bT4xNzg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjE3ODwvcmVjLW51bWJlcj48
Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTly
dXNwYXh4eHJmemZzdyI+MTc4PC9rZXk+PGtleSBhcHA9IkVOV2ViIiBkYi1pZD0iU2dSRGV3cnRx
Z2NBQUFwdk1DRSI+Njg1PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IldlYiBQ
YWdlIj4xMjwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkVkdWNhdGlv
bmFsIFRlc3RpbmcgU2VydmljZSw8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRp
dGxlcz48dGl0bGU+U2NvcmluZyBHdWlkZXMgKFJ1YnJpY3MpIGZvciBXcml0aW5nIFJlc3BvbnNl
cy48L3RpdGxlPjwvdGl0bGVzPjxudW1iZXI+QXByaWwgOCwgMjAwNzwvbnVtYmVyPjxkYXRlcz48
eWVhcj4yMDA0PC95ZWFyPjwvZGF0ZXM+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPnd3dy5ldHMu
b3JnIHVuZGVyICZxdW90O2lCVCBUT0VGTCBTY29yaW5nIEd1aWRlcyZxdW90OzwvdXJsPjwvcmVs
YXRlZC11cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5JbnRlcm5hdGlv
bmFsIEVuZ2xpc2ggTGFuZ3VhZ2UgVGVzdGluZyBTeXN0ZW08L0F1dGhvcj48WWVhcj5uLmQuPC9Z
ZWFyPjxSZWNOdW0+MjUxPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNTE8L3JlYy1udW1i
ZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBw
cDk5cnVzcGF4eHhyZnpmc3ciPjI1MTwva2V5PjxrZXkgYXBwPSJFTldlYiIgZGItaWQ9IlNnUkRl
d3J0cWdjQUFBcHZNQ0UiPjc1Mzwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJX
ZWIgUGFnZSI+MTI8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5JbnRl
cm5hdGlvbmFsIEVuZ2xpc2ggTGFuZ3VhZ2UgVGVzdGluZyBTeXN0ZW0sPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPklFTFRTIFRhc2sgMiB3cml0aW5nIGJh
bmQgZGVzY3JpcHRvcnMuPC90aXRsZT48L3RpdGxlcz48bnVtYmVyPkp1bmUgMTAsIDIwMDk8L251
bWJlcj48ZGF0ZXM+PHllYXI+bi5kLjwveWVhcj48L2RhdGVzPjx1cmxzPjxyZWxhdGVkLXVybHM+
PHVybD5odHRwOi8vd3d3LmllbHRzLm9yZy9yZXNlYXJjaGVycy9zY29yZV9wcm9jZXNzaW5nX2Fu
ZF9yZXBvcnRpbmcuYXNweDwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48L3JlY29yZD48L0Np
dGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Educational Testing Service, 2004; International English Language Testing System, n.d.-a, n.d.-b). It could also be argued that this trend may also be explained by increased phrasal elaboration and increased complexity that is measured by the words per clause ratio. However, that correlation was not statistically significant either, and with a value of .276 this trend is noticeably weaker than the trend observed for the clauses per T-unit ratio. Ultimately, only the errors per 100 words ratio shows a strong statistically significant correlation with the teacher-assigned grammar grade.One possible explanation is that increased levels of complexity may, in fact, be also associated with a higher number of errors. A statistically significant, moderate correlation between the clauses per T-unit ratio and the errors per 100 words ratio for Course 2 (r = .57, p < .05) provides support for this hypothesis. It appears that an increased error count, potentially as a result of more clauses per T-unit, may cause teachers to award a lower grammar grade despite higher levels of syntactic complexity. Consequently, the assessment criterion for accuracy seems to supersede any appreciation of complexity in L2 texts during the assessment process of grammatical ability in L2 writing. The correlations between the error per 100 words ratio and the words per clause ratio are not statistically significant in either course.The inclusion and the importance of the accuracy criterion for both teachers in this study also led to a conflict between instructional goals and the effect on the students. The accuracy criterion is included on the evaluation grid because higher proficiency in an L2 is associated with an increased control over or mastery of the structures of this language. Consequently, L2 writers should make fewer mistakes to reach higher levels on rating scales. The L2 teachers in this study focus on grammatical accuracy for this reason; they also wish to encourage their students to learn by pointing out particular areas of weakness (as Miriam, the teacher of Course 1, does) or through focus on sentence-level accuracy (as Elaine does in the weekly sentence writing practice). The goal for both teachers is to help students improve and learn to avoid making particular grammatical errors. This is clearly a positive instructional goal. This focus, however, also has a potentially negative result. First, students may actually take a rather pragmatic approach to meeting this criterion by avoiding risk-taking behaviour and employing safe language structures, thereby making fewer grammatical errors. Second, students may avoid expressing complex ideas that would require the use of more sophisticated or challenging linguistic structures so that they do not have to take a chance and risk grammatical errors, especially on important assignments or exams. As a result, teachers do not see the potential of what their students might be able to do (i.e., their full range of linguistic potential) but indirectly encourage the students to retreat to a linguistic safe zone. Ultimately, the focus on accuracy to the exclusion of complexity as an assessment criterion for grammatical ability in this study contravenes the goal in L2 pedagogy to encourage risk-taking. In L2 acquisition research, risk-taking has been found to be associated with L2 learning success PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Ccm93bjwvQXV0aG9yPjxZZWFyPjIwMDA8L1llYXI+PFJl
Y051bT4zNzQ8L1JlY051bT48RGlzcGxheVRleHQ+KEJyb3duLCAyMDAwOyBHYXNzICZhbXA7IFNl
bGlua2VyLCAyMDA4OyBMaWdodGJvd24gJmFtcDsgU3BhZGEsIDIwMTM7IE94Zm9yZCwgMTk5MDsg
U2tlaGFuLCAxOTg5KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4zNzQ8L3JlYy1u
dW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2Vh
djBwcDk5cnVzcGF4eHhyZnpmc3ciPjM3NDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu
YW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+QnJv
d24sIEguIERvdWdsYXM8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48
dGl0bGU+UHJpbmNpcGxlcyBvZiBsYW5ndWFnZSBsZWFybmluZyBhbmQgdGVhY2hpbmc8L3RpdGxl
PjwvdGl0bGVzPjxudW1iZXI+eGlpLCAzNTIgcC4gOjwvbnVtYmVyPjxlZGl0aW9uPjR0aDwvZWRp
dGlvbj48a2V5d29yZHM+PGtleXdvcmQ+TGFuZ3VhZ2UgYW5kIGxhbmd1YWdlcyBTdHVkeSBhbmQg
dGVhY2hpbmcuPC9rZXl3b3JkPjxrZXl3b3JkPkxhbmd1YWdlIGFjcXVpc2l0aW9uLjwva2V5d29y
ZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDAwPC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlv
bj5XaGl0ZSBQbGFpbnMsIE5ZPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5Mb25nbWFuPC9wdWJs
aXNoZXI+PGlzYm4+MDEzMDE3ODE2MCAoYWxrLiBwYXBlcik8L2lzYm4+PHVybHM+PC91cmxzPjwv
cmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkdhc3M8L0F1dGhvcj48WWVhcj4yMDA4PC9ZZWFy
PjxSZWNOdW0+MzcxPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zNzE8L3JlYy1udW1iZXI+
PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5
cnVzcGF4eHhyZnpmc3ciPjM3MTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJC
b29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+R2FzcywgU3Vz
YW4gTS48L2F1dGhvcj48YXV0aG9yPlNlbGlua2VyLCBMYXJyeTwvYXV0aG9yPjwvYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5TZWNvbmQgbGFuZ3VhZ2UgYWNxdWlzaXRpb246
IEFuIGludHJvZHVjdG9yeSBjb3Vyc2U8L3RpdGxlPjwvdGl0bGVzPjxudW1iZXI+NTkzIHAuPC9u
dW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPlNlY29uZCBsYW5ndWFnZSBhY3F1aXNpdGlvbi48L2tl
eXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwODwveWVhcj48L2RhdGVzPjxwdWItbG9j
YXRpb24+TmV3IFlvcms8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlJvdXRsZWRnZS9UYXlsb3Ig
JmFtcDsgRnJhbmNpczwvcHVibGlzaGVyPjxpc2JuPjk3ODAyMDM5MzI4NDEgKGVsZWN0cm9uaWMg
YmsuKSYjeEQ7MDIwMzkzMjg0NiAoZWxlY3Ryb25pYyBiay4pPC9pc2JuPjx1cmxzPjxyZWxhdGVk
LXVybHM+PHVybD5BY2Nlc3MgcmVzdHJpY3RlZCB0byBNY0dpbGwgdXNlcnMgaHR0cDovL3d3dy5u
ZXRMaWJyYXJ5LmNvbS91cmxhcGkuYXNwP2FjdGlvbj1zdW1tYXJ5JmFtcDt2PTEmYW1wO2Jvb2tp
ZD0yMTY5NjE8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRl
PjxBdXRob3I+TGlnaHRib3duPC9BdXRob3I+PFllYXI+MjAxMzwvWWVhcj48UmVjTnVtPjM5Njwv
UmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+Mzk2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+
PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3
Ij4zOTY8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5
cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkxpZ2h0Ym93biwgUGF0c3kgTS48L2F1
dGhvcj48YXV0aG9yPlNwYWRhLCBOaW5hPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPkhvdyBsYW5ndWFnZXMgYXJlIGxlYXJuZWQ8L3RpdGxlPjxzZWNvbmRh
cnktdGl0bGU+T3hmb3JkIGhhbmRib29rcyBmb3IgbGFuZ3VhZ2UgdGVhY2hlcnM8L3NlY29uZGFy
eS10aXRsZT48L3RpdGxlcz48bnVtYmVyPml4LCAyMzMgcC4gOjwvbnVtYmVyPjxlZGl0aW9uPjR0
aDwvZWRpdGlvbj48a2V5d29yZHM+PGtleXdvcmQ+TGFuZ3VhZ2UgYWNxdWlzaXRpb24uPC9rZXl3
b3JkPjxrZXl3b3JkPkNoaWxkcmVuIExhbmd1YWdlLjwva2V5d29yZD48a2V5d29yZD5MYW5ndWFn
ZSBhbmQgbGFuZ3VhZ2VzIFN0dWR5IGFuZCB0ZWFjaGluZy48L2tleXdvcmQ+PC9rZXl3b3Jkcz48
ZGF0ZXM+PHllYXI+MjAxMzwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+T3hmb3JkPC9wdWIt
bG9jYXRpb24+PHB1Ymxpc2hlcj5PeGZvcmQgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjxp
c2JuPjAxOTQ0MjIyNDAgKHBiay4pPC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+
PENpdGU+PEF1dGhvcj5PeGZvcmQ8L0F1dGhvcj48WWVhcj4xOTkwPC9ZZWFyPjxSZWNOdW0+Mzc1
PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zNzU8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5
cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpm
c3ciPjM3NTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYt
dHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+T3hmb3JkLCBSZWJlY2NhIEwuPC9h
dXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkxhbmd1YWdlIGxl
YXJuaW5nIHN0cmF0ZWdpZXM6IFdoYXQgZXZlcnkgdGVhY2hlciBzaG91bGQga25vdzwvdGl0bGU+
PC90aXRsZXM+PG51bWJlcj54eGlpLCAzNDIgcC4gOjwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29y
ZD5MYW5ndWFnZSBhbmQgbGFuZ3VhZ2VzIFN0dWR5IGFuZCB0ZWFjaGluZzwva2V5d29yZD48L2tl
eXdvcmRzPjxkYXRlcz48eWVhcj4xOTkwPC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5OZXcg
WW9yazwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+TmV3YnVyeSBIb3VzZTwvcHVibGlzaGVyPjxp
c2JuPjAwNjYzMjYwNzk8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48
QXV0aG9yPlNrZWhhbjwvQXV0aG9yPjxZZWFyPjE5ODk8L1llYXI+PFJlY051bT4zNzI8L1JlY051
bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM3MjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkg
YXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+Mzcy
PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Ta2VoYW4sIFBldGVyPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkluZGl2aWR1YWwgZGlmZmVyZW5jZXMg
aW4gc2Vjb25kLWxhbmd1YWdlIGxlYXJuaW5nPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNlY29u
ZCBsYW5ndWFnZSBhY3F1aXNpdGlvbi48L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48bnVtYmVy
PnZpLCAxNjggcC4gOjwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29yZD5TZWNvbmQgbGFuZ3VhZ2Ug
YWNxdWlzaXRpb24uPC9rZXl3b3JkPjxrZXl3b3JkPkluZGl2aWR1YWwgZGlmZmVyZW5jZXMuPC9r
ZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjE5ODk8L3llYXI+PC9kYXRlcz48cHViLWxv
Y2F0aW9uPkxvbmRvbjwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+QXJub2xkPC9wdWJsaXNoZXI+
PGlzYm4+MDcxMzE2NjAyOTwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5k
Tm90ZT5=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Ccm93bjwvQXV0aG9yPjxZZWFyPjIwMDA8L1llYXI+PFJl
Y051bT4zNzQ8L1JlY051bT48RGlzcGxheVRleHQ+KEJyb3duLCAyMDAwOyBHYXNzICZhbXA7IFNl
bGlua2VyLCAyMDA4OyBMaWdodGJvd24gJmFtcDsgU3BhZGEsIDIwMTM7IE94Zm9yZCwgMTk5MDsg
U2tlaGFuLCAxOTg5KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4zNzQ8L3JlYy1u
dW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2Vh
djBwcDk5cnVzcGF4eHhyZnpmc3ciPjM3NDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu
YW1lPSJCb29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+QnJv
d24sIEguIERvdWdsYXM8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48
dGl0bGU+UHJpbmNpcGxlcyBvZiBsYW5ndWFnZSBsZWFybmluZyBhbmQgdGVhY2hpbmc8L3RpdGxl
PjwvdGl0bGVzPjxudW1iZXI+eGlpLCAzNTIgcC4gOjwvbnVtYmVyPjxlZGl0aW9uPjR0aDwvZWRp
dGlvbj48a2V5d29yZHM+PGtleXdvcmQ+TGFuZ3VhZ2UgYW5kIGxhbmd1YWdlcyBTdHVkeSBhbmQg
dGVhY2hpbmcuPC9rZXl3b3JkPjxrZXl3b3JkPkxhbmd1YWdlIGFjcXVpc2l0aW9uLjwva2V5d29y
ZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDAwPC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlv
bj5XaGl0ZSBQbGFpbnMsIE5ZPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5Mb25nbWFuPC9wdWJs
aXNoZXI+PGlzYm4+MDEzMDE3ODE2MCAoYWxrLiBwYXBlcik8L2lzYm4+PHVybHM+PC91cmxzPjwv
cmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkdhc3M8L0F1dGhvcj48WWVhcj4yMDA4PC9ZZWFy
PjxSZWNOdW0+MzcxPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zNzE8L3JlYy1udW1iZXI+
PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5
cnVzcGF4eHhyZnpmc3ciPjM3MTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJC
b29rIj42PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+R2FzcywgU3Vz
YW4gTS48L2F1dGhvcj48YXV0aG9yPlNlbGlua2VyLCBMYXJyeTwvYXV0aG9yPjwvYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5TZWNvbmQgbGFuZ3VhZ2UgYWNxdWlzaXRpb246
IEFuIGludHJvZHVjdG9yeSBjb3Vyc2U8L3RpdGxlPjwvdGl0bGVzPjxudW1iZXI+NTkzIHAuPC9u
dW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPlNlY29uZCBsYW5ndWFnZSBhY3F1aXNpdGlvbi48L2tl
eXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwODwveWVhcj48L2RhdGVzPjxwdWItbG9j
YXRpb24+TmV3IFlvcms8L3B1Yi1sb2NhdGlvbj48cHVibGlzaGVyPlJvdXRsZWRnZS9UYXlsb3Ig
JmFtcDsgRnJhbmNpczwvcHVibGlzaGVyPjxpc2JuPjk3ODAyMDM5MzI4NDEgKGVsZWN0cm9uaWMg
YmsuKSYjeEQ7MDIwMzkzMjg0NiAoZWxlY3Ryb25pYyBiay4pPC9pc2JuPjx1cmxzPjxyZWxhdGVk
LXVybHM+PHVybD5BY2Nlc3MgcmVzdHJpY3RlZCB0byBNY0dpbGwgdXNlcnMgaHR0cDovL3d3dy5u
ZXRMaWJyYXJ5LmNvbS91cmxhcGkuYXNwP2FjdGlvbj1zdW1tYXJ5JmFtcDt2PTEmYW1wO2Jvb2tp
ZD0yMTY5NjE8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRl
PjxBdXRob3I+TGlnaHRib3duPC9BdXRob3I+PFllYXI+MjAxMzwvWWVhcj48UmVjTnVtPjM5Njwv
UmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+Mzk2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+
PGtleSBhcHA9IkVOIiBkYi1pZD0iZnhyZjBkOTV2d2VmNXdlYXYwcHA5OXJ1c3BheHh4cmZ6ZnN3
Ij4zOTY8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iQm9vayI+NjwvcmVmLXR5
cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkxpZ2h0Ym93biwgUGF0c3kgTS48L2F1
dGhvcj48YXV0aG9yPlNwYWRhLCBOaW5hPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz
Pjx0aXRsZXM+PHRpdGxlPkhvdyBsYW5ndWFnZXMgYXJlIGxlYXJuZWQ8L3RpdGxlPjxzZWNvbmRh
cnktdGl0bGU+T3hmb3JkIGhhbmRib29rcyBmb3IgbGFuZ3VhZ2UgdGVhY2hlcnM8L3NlY29uZGFy
eS10aXRsZT48L3RpdGxlcz48bnVtYmVyPml4LCAyMzMgcC4gOjwvbnVtYmVyPjxlZGl0aW9uPjR0
aDwvZWRpdGlvbj48a2V5d29yZHM+PGtleXdvcmQ+TGFuZ3VhZ2UgYWNxdWlzaXRpb24uPC9rZXl3
b3JkPjxrZXl3b3JkPkNoaWxkcmVuIExhbmd1YWdlLjwva2V5d29yZD48a2V5d29yZD5MYW5ndWFn
ZSBhbmQgbGFuZ3VhZ2VzIFN0dWR5IGFuZCB0ZWFjaGluZy48L2tleXdvcmQ+PC9rZXl3b3Jkcz48
ZGF0ZXM+PHllYXI+MjAxMzwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+T3hmb3JkPC9wdWIt
bG9jYXRpb24+PHB1Ymxpc2hlcj5PeGZvcmQgVW5pdmVyc2l0eSBQcmVzczwvcHVibGlzaGVyPjxp
c2JuPjAxOTQ0MjIyNDAgKHBiay4pPC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+
PENpdGU+PEF1dGhvcj5PeGZvcmQ8L0F1dGhvcj48WWVhcj4xOTkwPC9ZZWFyPjxSZWNOdW0+Mzc1
PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zNzU8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5
cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJmeHJmMGQ5NXZ3ZWY1d2VhdjBwcDk5cnVzcGF4eHhyZnpm
c3ciPjM3NTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJCb29rIj42PC9yZWYt
dHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+T3hmb3JkLCBSZWJlY2NhIEwuPC9h
dXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkxhbmd1YWdlIGxl
YXJuaW5nIHN0cmF0ZWdpZXM6IFdoYXQgZXZlcnkgdGVhY2hlciBzaG91bGQga25vdzwvdGl0bGU+
PC90aXRsZXM+PG51bWJlcj54eGlpLCAzNDIgcC4gOjwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29y
ZD5MYW5ndWFnZSBhbmQgbGFuZ3VhZ2VzIFN0dWR5IGFuZCB0ZWFjaGluZzwva2V5d29yZD48L2tl
eXdvcmRzPjxkYXRlcz48eWVhcj4xOTkwPC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5OZXcg
WW9yazwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+TmV3YnVyeSBIb3VzZTwvcHVibGlzaGVyPjxp
c2JuPjAwNjYzMjYwNzk8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48
QXV0aG9yPlNrZWhhbjwvQXV0aG9yPjxZZWFyPjE5ODk8L1llYXI+PFJlY051bT4zNzI8L1JlY051
bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM3MjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkg
YXBwPSJFTiIgZGItaWQ9ImZ4cmYwZDk1dndlZjV3ZWF2MHBwOTlydXNwYXh4eHJmemZzdyI+Mzcy
PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkJvb2siPjY8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Ta2VoYW4sIFBldGVyPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkluZGl2aWR1YWwgZGlmZmVyZW5jZXMg
aW4gc2Vjb25kLWxhbmd1YWdlIGxlYXJuaW5nPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNlY29u
ZCBsYW5ndWFnZSBhY3F1aXNpdGlvbi48L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48bnVtYmVy
PnZpLCAxNjggcC4gOjwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29yZD5TZWNvbmQgbGFuZ3VhZ2Ug
YWNxdWlzaXRpb24uPC9rZXl3b3JkPjxrZXl3b3JkPkluZGl2aWR1YWwgZGlmZmVyZW5jZXMuPC9r
ZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjE5ODk8L3llYXI+PC9kYXRlcz48cHViLWxv
Y2F0aW9uPkxvbmRvbjwvcHViLWxvY2F0aW9uPjxwdWJsaXNoZXI+QXJub2xkPC9wdWJsaXNoZXI+
PGlzYm4+MDcxMzE2NjAyOTwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjwvRW5k
Tm90ZT5=
ADDIN EN.CITE.DATA (Brown, 2000; Gass & Selinker, 2008; Lightbown & Spada, 2013; Oxford, 1990; Skehan, 1989). For this reason, language teaching methodology books ADDIN EN.CITE <EndNote><Cite><Author>Brown</Author><Year>2007</Year><RecNum>400</RecNum><Prefix>e.g.`, </Prefix><DisplayText>(e.g., Brown, 2007)</DisplayText><record><rec-number>400</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">400</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Brown, H. Douglas</author></authors></contributors><titles><title>Teaching by principles: An interactive approach to language pedagogy</title></titles><number>xvi, 480 p. :</number><edition>3rd</edition><keywords><keyword>Language and languages Study and teaching.</keyword><keyword>English language Study and teaching Foreign speakers.</keyword><keyword>Interaction analysis in education.</keyword></keywords><dates><year>2007</year></dates><pub-location>White Plains, NY</pub-location><publisher>Pearson</publisher><isbn>0130282839 (pbk.)</isbn><urls></urls></record></Cite></EndNote>(e.g., Brown, 2007) and language curricula ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Ministère d'?ducation</Author><Year>n.d.</Year><RecNum>376</RecNum><Prefix>e.g.`, Ministère d'?ducation`, Loisir et Sport`, </Prefix><DisplayText>(e.g., Ministère d'?ducation, Loisir et Sport, n.d.)</DisplayText><record><rec-number>376</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">376</key></foreign-keys><ref-type name="Electronic Book">44</ref-type><contributors><authors><author>Ministère d'?ducation, , Loisir et Sport</author></authors></contributors><titles><title>Programme de formation de l'école québécoise: Enseignment secondaire, deuxième cycle (English as a Second Language). [Quebec education program: Secondary school education, cycle two]</title></titles><dates><year>n.d.</year></dates><urls><related-urls><url>;(e.g., Ministère d'?ducation, Loisir et Sport, n.d.) recommend that teachers encourage student risk-taking behaviour to facilitate and promote language learning in the classroom. The data show that the teachers in this study had difficulty balancing the assessment criteria for grammatical accuracy and complexity. This finding is similar to what Fritz and Ruegg ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Fritz</Author><Year>2013</Year><RecNum>474</RecNum><DisplayText>(2013)</DisplayText><record><rec-number>474</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">474</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Fritz, Erik</author><author>Ruegg, Rachael</author></authors></contributors><titles><title>Rater sensitivity to lexical accuracy, sophistication and range when assessing writing</title><secondary-title>Assessing Writing</secondary-title></titles><periodical><full-title>Assessing Writing</full-title></periodical><pages>173-181</pages><volume>18</volume><number>2</number><keywords><keyword>Lexis</keyword><keyword>Writing</keyword><keyword>Lexical accuracy</keyword><keyword>Lexical range</keyword><keyword>Lexical sophistication</keyword></keywords><dates><year>2013</year><pub-dates><date>4//</date></pub-dates></dates><isbn>1075-2935</isbn><urls><related-urls><url> this study to point out that they had a similar finding for complexity/range as I did for grammar. Cite what they say (look for cliche of correcting errors).</research-notes></record></Cite></EndNote>(2013) found in their study of the rating of vocabulary in writing. Although raters had been trained to assess accuracy as well as range, scores were only related to lexical accuracy but not range. Achieving a balance between grammatical accuracy and complexity is just as important in L2 writing assessment as Li ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Li</Author><Year>2000</Year><RecNum>162</RecNum><DisplayText>(2000)</DisplayText><record><rec-number>162</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">162</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">669</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Li, Yili</author></authors></contributors><titles><title>Linguistic characteristics of ESL writing in task-based e-mail activities</title><secondary-title>System</secondary-title></titles><periodical><full-title>System</full-title></periodical><pages>229-245</pages><volume>28</volume><number>2</number><dates><year>2000</year></dates><urls></urls><electronic-resource-num>10.1016/S0346-251X(00)00009-9 </electronic-resource-num></record></Cite></EndNote>(2000) had argued for L2 writing instruction. For this reason, some developers structure rating scales so that L2 test takers are not penalized for taking risks but are, in fact, encouraged to show the full range of their linguistic abilities. This approach has been adopted for the rating scales of the Deutsche Sprachdiplom (German Language Diploma, GLD) with separate criteria for linguistic accuracy and linguistic range or repertoire. For the criterion Strukturen (structures), raters judge the level of complexity and the linguistic repertoire in texts, whereas they assess the level of morphosyntactic accuracy under the criterion Korrektheit (accuracy) ADDIN EN.CITE <EndNote><Cite><Author>Zentralstelle für das Auslandsschulwesen</Author><Year>2012</Year><RecNum>384</RecNum><DisplayText>(Zentralstelle für das Auslandsschulwesen, 2012)</DisplayText><record><rec-number>384</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">384</key></foreign-keys><ref-type name="Electronic Book">44</ref-type><contributors><authors><author>Zentralstelle für das Auslandsschulwesen,</author></authors></contributors><titles><title>Deutsches Sprachdiplom der Kultusministerkonferenz: Handreichungen für die schrifltliche Kommunikation (Niveaustufe B2/C1). [German language diploma of the education ministers' conference: Documents for the written communication (Level B2/C1)]</title></titles><dates><year>2012</year></dates><urls><related-urls><url>;(Zentralstelle für das Auslandsschulwesen, 2012). Furthermore, the GLD handbook specifically discusses the relationship between these two criteria and how raters should address them jointly. This directive is based on the principle that learners’ risk-taking should be rewarded and not discouraged so that a negative washback on instruction can be avoided ADDIN EN.CITE <EndNote><Cite><Author>Zentralstelle für das Auslandsschulwesen</Author><Year>2012</Year><RecNum>384</RecNum><DisplayText>(Zentralstelle für das Auslandsschulwesen, 2012)</DisplayText><record><rec-number>384</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">384</key></foreign-keys><ref-type name="Electronic Book">44</ref-type><contributors><authors><author>Zentralstelle für das Auslandsschulwesen,</author></authors></contributors><titles><title>Deutsches Sprachdiplom der Kultusministerkonferenz: Handreichungen für die schrifltliche Kommunikation (Niveaustufe B2/C1). [German language diploma of the education ministers' conference: Documents for the written communication (Level B2/C1)]</title></titles><dates><year>2012</year></dates><urls><related-urls><url>;(Zentralstelle für das Auslandsschulwesen, 2012).Implications: The Assessment of Grammatical Ability in Practice.The assessment of grammatical ability in academic writing is an important component of the overall assessment of students’ ability to write in an L2. The definition of grammar and grammatical ability as a construct in the literature is relatively complex, but the findings of this study indicate that the construct assessed in the classroom by academic L2 writing teachers is reduced to accuracy on the sentence level. Although Elaine, the teacher for Course 2, and certain students in this study refer to complexity and variety as an assessment criterion, the text-based analysis does not provide confirmation for this. In fact, if anything, there appears to be an opposite trend in the data for Course 2. There appears to be evidence then that the construct assessed by the teachers does not resemble the model put forward in the testing literature. For this reason, the assessment of grammatical ability is based on what Messick ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Messick</Author><Year>1989</Year><RecNum>294</RecNum><DisplayText>(1989)</DisplayText><record><rec-number>294</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">294</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">794</key></foreign-keys><ref-type name="Book Section">5</ref-type><contributors><authors><author>Messick, Samuel</author></authors><secondary-authors><author>Linn, Robert L.</author></secondary-authors></contributors><titles><title>Validity</title><secondary-title>Educational measurement</secondary-title></titles><pages>13-103</pages><edition>3rd</edition><dates><year>1989</year></dates><pub-location>New York</pub-location><publisher>American Council on Education/Macmillan</publisher><urls></urls></record></Cite></EndNote>(1989) calls an underrepresentation of the theoretical construct. This finding should be cause for concern because of the mismatch between the labelling of the construct (grammar) and its operationalization (accuracy only). It appears that for the teachers in this study grammar equals accuracy whereas the language testing literature defines the construct more broadly and comprehensively. For rating scale developers, the findings of this study mean that, despite all best intentions, the rating scale may not be used as planned and a different construct is actually assessed. The structure of a rating scale may lead to a simplified construct operationalization of grammatical ability in practice despite a complex operationalization of the construct in the rating scale development process when grammatical accuracy, grammatical complexity, and effectiveness and variety of syntactic constructions are included under one criterion. If grammatical accuracy, however, is assessed separately from other aspects of grammatical ability, as Fritz and Ruegg ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Fritz</Author><Year>2013</Year><RecNum>474</RecNum><DisplayText>(2013)</DisplayText><record><rec-number>474</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">474</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Fritz, Erik</author><author>Ruegg, Rachael</author></authors></contributors><titles><title>Rater sensitivity to lexical accuracy, sophistication and range when assessing writing</title><secondary-title>Assessing Writing</secondary-title></titles><periodical><full-title>Assessing Writing</full-title></periodical><pages>173-181</pages><volume>18</volume><number>2</number><keywords><keyword>Lexis</keyword><keyword>Writing</keyword><keyword>Lexical accuracy</keyword><keyword>Lexical range</keyword><keyword>Lexical sophistication</keyword></keywords><dates><year>2013</year><pub-dates><date>4//</date></pub-dates></dates><isbn>1075-2935</isbn><urls><related-urls><url> this study to point out that they had a similar finding for complexity/range as I did for grammar. Cite what they say (look for cliche of correcting errors).</research-notes></record></Cite></EndNote>(2013) suggest for vocabulary based on the findings of their study, rating scale designers could ensure positive washback through writing assessment to promote learning more broadly rather than generate a focus on error avoidance strategies by the students. Maybe this emphasis would help both L2 writing teachers and their students to move away from “an exaggerated emphasis on grammatical accuracy” ADDIN EN.CITE <EndNote><Cite><Author>Iwashita</Author><Year>2008</Year><RecNum>216</RecNum><Pages>47</Pages><DisplayText>(Iwashita et al., 2008, p. 47)</DisplayText><record><rec-number>216</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">216</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">719</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Iwashita, Noriko</author><author>Brown, Annie</author><author>McNamara, Tim</author><author>O'Hagan, Sally</author></authors></contributors><titles><title>Assessed levels of second language speaking proficiency: How distinct?</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>24-49</pages><volume>29</volume><number>1</number><dates><year>2008</year><pub-dates><date>March 1, 2008</date></pub-dates></dates><urls><related-urls><url> </url></related-urls></urls><electronic-resource-num>10.1093/applin/amm017</electronic-resource-num></record></Cite></EndNote>(Iwashita et al., 2008, p. 47). This would potentially allow the students to deal with language more creatively and to have the room to experiment with the language so that their linguistic abilities could grow and develop. It would, therefore, be worth considering the adoption of this approach to rating scale design more widely, especially for classroom assessment.For teacher educators, these results have implications in terms of what students should learn in their courses. It seems that a concerted effort may be required by teachers to resist the temptation to reduce grammar in writing to accuracy only. Rating scales that separate the constructs of accuracy and complexity may help teachers to do that ADDIN EN.CITE <EndNote><Cite><Author>Zentralstelle für das Auslandsschulwesen</Author><Year>2012</Year><RecNum>384</RecNum><Prefix>e.g.`, the GLD rating scales`, </Prefix><DisplayText>(e.g., the GLD rating scales, Zentralstelle für das Auslandsschulwesen, 2012)</DisplayText><record><rec-number>384</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">384</key></foreign-keys><ref-type name="Electronic Book">44</ref-type><contributors><authors><author>Zentralstelle für das Auslandsschulwesen,</author></authors></contributors><titles><title>Deutsches Sprachdiplom der Kultusministerkonferenz: Handreichungen für die schrifltliche Kommunikation (Niveaustufe B2/C1). [German language diploma of the education ministers' conference: Documents for the written communication (Level B2/C1)]</title></titles><dates><year>2012</year></dates><urls><related-urls><url>;(e.g., the GLD rating scales, Zentralstelle für das Auslandsschulwesen, 2012), but this would have to be explored in future research. Finally, teachers may not have considered how their marking criterion with its focus on accuracy affects the strategies students employ to satisfy their teacher’s expectations. Teachers, consequently, may not see what students can do, but only what students think teachers want them to produce. This raises the question to what extent the evaluation grid used in this ESL program is useful in assessing grammatical ability. Does this scale hinder or assist teachers in the comprehensive assessment of grammatical ability in academic writing? It seems the former may be the case.Finally, L2 writing teachers probably need to find a way that allows them to promote grammatical accuracy and increased complexity at the same time. If that is not possible, it may be worth exploring if the assessment focus in terms of grammar can shift depending on the writing task or assignment, which would, of course, have to be clearly communicated to the students. Should program and/or university policy not permit teachers to do this, ungraded work could be used to encourage students to take risks and try out new and/or more complex linguistic structures with which they are less familiar. This may be realized, for example, by not grading first drafts, as Elaine did in this study. Either of these practices would provide a space for students in which they could safely experiment with language and push their limits without fearing the reprisal of grades and the loss of points for every grammatical mistake that they make.Strengths and Limitations of the StudyThis study has two strengths that may be considered limitations by some. First, the findings are based on a case study of 2 teachers and their 33 students in one particular educational context. Although this context is similar to other educational settings in terms of instructional goals, purpose of the program, and its student clientele, the findings may be a product of the interaction between the research participants in this study and therefore not valid beyond the current research context. Yin ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Yin</Author><Year>2009</Year><RecNum>370</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>370</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">370</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Yin, Robert K.</author></authors></contributors><titles><title>Case study research: Design and methods</title><secondary-title>Applied social research methods series ; 5</secondary-title></titles><edition>4th</edition><keywords><keyword>Case method.</keyword><keyword>Social sciences Research Methodology.</keyword></keywords><dates><year>2009</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>9781412960991 (pbk.)
1412960991 (pbk.)</isbn><urls></urls></record></Cite></EndNote>(2009) argues, however, the findings of any empirical study (and not just case studies) have to be confirmed by further research. Second, the study is based on a mixed methods research design. Such a design “offsets the weaknesses of both quantitative and qualitative research” ADDIN EN.CITE <EndNote><Cite><Author>Creswell</Author><Year>2007</Year><RecNum>187</RecNum><Pages>9</Pages><DisplayText>(Creswell & Plano Clark, 2007, p. 9)</DisplayText><record><rec-number>187</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">187</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">691</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Creswell, John W.</author><author>Plano Clark, Vicki L.</author></authors></contributors><titles><title>Designing and conducting mixed methods research</title></titles><keywords><keyword>Social sciences Research Methodology.</keyword><keyword>Research Evaluation.</keyword></keywords><dates><year>2007</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>1412927919 (cloth)
1412927927 (pbk.)
9781412927918
9781412927925</isbn><urls><related-urls><url>Table of contents only description </url></related-urls></urls></record></Cite></EndNote>(Creswell & Plano Clark, 2007, p. 9) but also requires the merging of different data sources and the consolidation of potentially contradictory findings ADDIN EN.CITE <EndNote><Cite><Author>Creswell</Author><Year>2007</Year><RecNum>187</RecNum><DisplayText>(Creswell & Plano Clark, 2007; Teddlie & Tashakkori, 2009)</DisplayText><record><rec-number>187</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">187</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">691</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Creswell, John W.</author><author>Plano Clark, Vicki L.</author></authors></contributors><titles><title>Designing and conducting mixed methods research</title></titles><keywords><keyword>Social sciences Research Methodology.</keyword><keyword>Research Evaluation.</keyword></keywords><dates><year>2007</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>1412927919 (cloth)
1412927927 (pbk.)
9781412927918
9781412927925</isbn><urls><related-urls><url>Table of contents only description </url></related-urls></urls></record></Cite><Cite><Author>Teddlie</Author><Year>2009</Year><RecNum>292</RecNum><record><rec-number>292</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">292</key><key app="ENWeb" db-id="SgRDewrtqgcAAApvMCE">792</key></foreign-keys><ref-type name="Book">6</ref-type><contributors><authors><author>Teddlie, Charles</author><author>Tashakkori, Abbas</author></authors></contributors><titles><title>Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences</title></titles><keywords><keyword>Social sciences Research Methodology.</keyword></keywords><dates><year>2009</year></dates><pub-location>Thousand Oaks, CA</pub-location><publisher>SAGE</publisher><isbn>9780761930112 (cloth)
0761930116 (cloth)
9780761930129 (pbk.)
0761930124 (pbk.)</isbn><urls></urls></record></Cite></EndNote>(Creswell & Plano Clark, 2007; Teddlie & Tashakkori, 2009). Depending on one’s worldview, this can be a strength or a limitation. The results of the current study, however, provide substantive evidence of the usefulness of the design because the overall finding for grammatical accuracy as an important assessment criterion for grammatical ability is confirmed by different data sources across the two courses.Another limitation of this study is the context within which it was conducted. This study took place in university academic writing classes with a focus on developing students’ ability to write academic essays, and the students in this study are fairly proficient users of ESL. Although the essay exams analyzed in this study are not the only writing task that students in these classes complete, the focus of these courses is on essay writing tasks. The findings of this study might be different in writing courses that include a wider range of writing tasks (e.g., research papers) or have student participants from other age groups and/or with higher and/or lower proficiency levels.ConclusionThe current study of classroom assessment of grammar in L2 academic writing has two important findings. First, this case study discovered a gap between the theoretical definition of grammatical ability in the language testing literature and the construct that is assessed in the L2 writing classrooms in this study, despite the teachers’ attempts to assess grammatical ability more broadly. Second, the findings reveal that even classroom assessment creates a washback effect on how and what students may learn. Alderson and Wall ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Alderson</Author><Year>1993</Year><RecNum>434</RecNum><DisplayText>(1993)</DisplayText><record><rec-number>434</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">434</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Alderson, J. Charles</author><author>Wall, Dianne</author></authors></contributors><titles><title>Does washback exist?</title><secondary-title>Applied Linguistics</secondary-title></titles><periodical><full-title>Applied Linguistics</full-title></periodical><pages>115-129</pages><volume>14</volume><number>2</number><dates><year>1993</year><pub-dates><date>June 1, 1993</date></pub-dates></dates><urls><related-urls><url>;(1993) formulated possible washback hypotheses, one of which states that “a test will influence what learners learn; and a test will influence how learners learn” (p. 120). This is not new to teachers who consciously use tests and other forms of assessment in an attempt to positively influence student behaviour ADDIN EN.CITE <EndNote><Cite><Author>Wall</Author><Year>1993</Year><RecNum>435</RecNum><DisplayText>(Wall & Alderson, 1993)</DisplayText><record><rec-number>435</rec-number><foreign-keys><key app="EN" db-id="fxrf0d95vwef5weav0pp99ruspaxxxrfzfsw">435</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Wall, Dianne</author><author>Alderson, J. Charles</author></authors></contributors><titles><title>Examining washback: The Sri Lankan impact study</title><secondary-title>Language Testing</secondary-title></titles><periodical><full-title>Language Testing</full-title></periodical><pages>41-69</pages><volume>10</volume><number>1</number><dates><year>1993</year><pub-dates><date>March 1, 1993</date></pub-dates></dates><urls><related-urls><url>;(Wall & Alderson, 1993). This study revealed, however, that the relationship between the teachers’ pedagogical goals, assessment criteria, and learning objectives for students and the resulting student behaviour is not necessarily straightforward. The teachers in this study want their students to improve their grammatical ability, so the teachers encourage them to increase the level of grammatical accuracy in their writing. As the findings from the interviews with students in this study indicate, however, students may try to meet these objectives in ways not intended by the teacher: a focus on error avoidance instead of risk-taking, the latter of which is associated with L2 learning.AcknowledgementsThis study was undertaken in partial fulfillment of the requirements of the author’s PhD in Second Language Education at McGill University. Earlier versions of this paper were presented at the Symposium on Second Language Writing in Phoenix, AZ in November 2009 and at the Language Testing Research Colloquium in Cambridge, United Kingdom in April 2010. The author would like to thank Alison Crump, Bonnie Barnett, Kerri Staples, and May Tan for their help in data transcription and analysis, and Carolyn E. Turner, Beverly A. Baker and the anonymous reviewers for their helpful comments on earlier versions of this paper. This study was supported by a doctoral bursary from the FQRSC (Fonds québécois de la recherche sur la société et la culture).References ADDIN EN.REFLIST Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14(2), 115-129. doi: 10.1093/applin/14.2.115Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.Baker, B. A. (2010). In the service of the stakeholder: A critical, mixed methods program of research in high stakes language assessment. (Unpublished doctoral dissertation). McGill University, Montreal, Quebec, Canada, Montreal. Retrieved from Bardovi-Harlig, K., & Bofman, T. (1989). Attainment of syntactic and morphological accuracy by advanced language learners. Studies in Second Language Acquisition, 11(1), 17-34. doi: 10.1017/S0272263100007816Barkaoui, K. (2010a). Explaining ESL essay holistic scores: A multilevel modeling approach. Language Testing, 27(4), 515-535. doi: 10.1177/0265532210368717Barkaoui, K. (2010b). Variability in ESL essay rating processes: The role of the rating scale and rater experience. Language Assessment Quarterly, 7(1), 54 - 74. doi: 10.1080/15434300903464418 Beers, S., & Nagy, W. (2009). Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre? Reading and Writing, 22(2), 185-200. doi: 10.1007/s11145-007-9107-5Biber, D. (2006). University language: A corpus-based study of spoken and written registers. Amsterdam: John Benjamins.Brown, H. D. (2000). Principles of language learning and teaching (4th ed.). White Plains, NY: Longman.Brown, H. D. (2007). Teaching by principles: An interactive approach to language pedagogy (3rd ed.). White Plains, NY: Pearson.Byrnes, H. (2002). The role of task and task-based assessment in a content-oriented collegiate foreign language curriculum. Language Testing, 19(4), 419-437. doi: 10.1191/0265532202lt238oaByrnes, H., Maxim, H. H., & Norris, J. M. (2010). Realizing advanced FL writing development in collegiate education: Curricular design, pedagogy, assessment. Modern Language Journal, 94(Monograph Issue). doi: 10.1111/j.1540-4781.2010.01147.xCanale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1-47. doi: 10.1093/applin/I.1.1 Charmaz, K. (2003). Qualitative interviewing and grounded theory analysis. In J. A. Holstein & J. F. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 311-330). Thousand Oaks, CA: SAGE.Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: SAGE.Corbin, J. M., & Strauss, A. L. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks, CA: SAGE.Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Retrieved from Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: SAGE.Creswell, J. W., & Miller, D. L. (2000). Determining validity in qualitative inquiry. Theory into Practice, 39(3), 124-130. doi: 10.1207/s15430421tip3903_2Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: SAGE.Crusan, D. (2010). Assessment in the second language writing classroom. Ann Arbor, MI: The University of Michigan Press.Cumming, A. (2004). Broadening, deepening, and consolidating. Language Assessment Quarterly, 1(1), 5-18. doi: 10.1207/s15434311laq0101_2 Cumming, A. (2013). Assessing integrated writing tasks for academic purposes: Promises and Perils. Language Assessment Quarterly, 10(1), 1-8. doi: 10.1080/15434303.2011.622016Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10(1), 5-43. doi: 10.1016/j.asw.2005.02.001Educational Testing Service. (2004). Scoring Guides (Rubrics) for Writing Responses. Retrieved April 8, 2007, from under "iBT TOEFL Scoring Guides"Ellis, R., & Yuan, F. (2004). The effects of planning on fluency, complexity, and accuracy in second language narrative writing. Studies in Second Language Acquisition, 26(1), 59-84. doi: 10.1017/S0272263104026130Ferris, D. R. (2002). Treatment of error in second language student writing. Ann Arbor: University of Michigan Press.Ferris, D. R. (2003). Response to student writing: Implications for second language students. Mahwah, N.J.: Lawrence Erlbaum Associates.Ferris, D. R. (2007). Preparing teachers to respond to student writing. Journal of Second Language Writing, 16(3), 165-193. doi: 10.1016/j.jslw.2007.07.003 Ferris, D. R., & Hedgcock, J. (2005). Teaching ESL composition: Purpose, process, and practice. Mahwah, N.J.; London: Lawrence Erlbaum.Fritz, E., & Ruegg, R. (2013). Rater sensitivity to lexical accuracy, sophistication and range when assessing writing. Assessing Writing, 18(2), 173-181. doi: , S. M., & Selinker, L. (2008). Second language acquisition: An introductory course. New York: Routledge/Taylor & Francis.Grabe, W., & Kaplan, R. B. (1996). Theory & practice of writing: An applied linguistic perspective. New York: Addison Wesley Longman.Halliday, M. A. K., & Martin, J. R. (1993). Writing science: Literacy and discursive power. Pittsburgh: University of Pittsburgh Press.Halliday, M. A. K., & Matthiessen, C. M. I. M. (2004). An introduction to functional grammar (3rd ed.). London: Arnold.Hamp-Lyons, L. (1991). What is a "writing test"? In L. Hamp-Lyons (Ed.), Assessing second language writing in academic contexts (pp. 5-15). Norwood, NJ: Ablex.Hamp-Lyons, L. (2011). Writing assessment: Shifting issues, new tools, enduring questions. Assessing Writing, 16(1), 3-5. doi: , J. R. (1996). A new framework for understanding cognition and affect in writing. In C. M. Levy & S. Randsdell (Eds.), The science of writing: Theories, methods, individual differences, and applications (pp. 1-27). Mahwah, NJ: Lawrence Erlbaum.Hayes, J. R., & Flower, L. S. (1980). Identifying the organization of writing processes. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 3-30). Hillsdale, NJ: Erlbaum.He, L., & Shi, L. (2012). Topical knowledge and ESL writing. Language Testing, 29(3), 443-464. doi: 10.1177/0265532212436659Homburg, T. J. (1984). Holistic evaluation of ESL compositions: Can it be validated objectively? TESOL Quarterly, 18(1), 87-107. Housen, A., & Kuiken, F. (2009a). Complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30(4), 461-473. doi: 10.1093/applin/amp048Housen, A., & Kuiken, F. (2009b). Complexity, accuracy, and fluency in second language acquisition [Special issue]. Applied Linguistics, 30(4), 461 - 626 Huang, J., & Foote, C. J. (2010). Grading between the lines: What really impacts professors’ holistic evaluation of ESL graduate student writing? Language Assessment Quarterly, 7(3), 219-233. doi: 10.1080/15434300903540894Hunt, K. W. (1965). Grammatical structures written at three grade levels. Urbana, IL: The National Council of Teachers of English.Hyland, K. (2006). English for academic purposes: An advanced resource book. London: Routledge.International English Language Testing System. (n.d.-a). IELTS Task 1 writing band descriptors. Retrieved June 10, 2009, from English Language Testing System. (n.d.-b). IELTS Task 2 writing band descriptors. Retrieved June 10, 2009, from , N., Brown, A., McNamara, T., & O'Hagan, S. (2008). Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29(1), 24-49. doi: 10.1093/applin/amm017Kuiken, F., & Vedder, I. (2008). Cognitive task complexity and written output in Italian and French as a foreign language. Journal of Second Language Writing, 17(1), 48-60. doi: 10.1016/j.jslw.2007.08.003 Leki, I. (2001). Hearing voices: L2 students' experiences in L2 writing courses. In T. Silva & P. K. Matsuda (Eds.), On second language writing (pp. 17-28). Mahwah, NJ: Lawrence Erlbaum.Li, Y. (2000). Linguistic characteristics of ESL writing in task-based e-mail activities. System, 28(2), 229-245. doi: 10.1016/S0346-251X(00)00009-9 Lightbown, P. M., & Spada, N. (2013). How languages are learned (4th ed.). Oxford: Oxford University Press.Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: SAGE.Lumley, T. (2005). Assessing second language writing: The rater's perspective. Frankfurt: Peter Lang.McNamara, T. F. (1996). Measuring second language performance. New York: Longman.McNamara, T. F. (2001). Rethinking alternative assessment. Language Testing, 18(4), 329-332. doi: 10.1177/026553220101800401Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: American Council on Education/Macmillan.Ministère d'?ducation, Loisir et Sport. (n.d.). Programme de formation de l'école québécoise: Enseignment secondaire, deuxième cycle (English as a Second Language). [Quebec education program: Secondary school education, cycle two] Retrieved from Neumann, H. (2011). What's in a grade? A mixed methods investigation of teacher assessment of grammatical ability in L2 academic writing. (Unpublished doctoral dissertation). McGill University, Montreal. Retrieved from Norris, J. M., Brown, J. D., Hudson, T., & Yoshioka, J. (1998). Designing second language performance assessments. Honolulu, HI: University of Hawai'i, Second Language Teaching and Curriculum Center.Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555-578. doi: 10.1093/applin/amp044Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492-518. doi: 10.1093/applin/24.4.492 Oxford, R. L. (1990). Language learning strategies: What every teacher should know. New York: Newbury House.Polio, C. G. (1997). Measures of linguistic accuracy in second language writing research. Language Learning, 47(1), 101-143. doi: 10.1111/0023-8333.31997003Polio, C. G., & Williams, J. (2009). Teaching and Testing Writing. In M. H. Long & C. J. Doughty (Eds.), The Handbook of Language Teaching (pp. 486-517): Wiley-Blackwell.Purpura, J. E. (2004). Assessing grammar. Cambridge: Cambridge University Press.Quirk, R., Greenbaum, S., Leech, G., & Svartvik, J. (1985). A comprehensive grammar of the English language. London & New York: Longman.Rea-Dickins, P. (1997). So, why do we need relationships with stakeholders in language testing? A view from the UK. Language Testing, 14(3), 304-314. doi: 10.1177/026553229701400307Rea-Dickins, P. (2009). Classroom-based assessment. In E. Shohamy & N. H. Hornberger (Eds.), Encyclopedia of language and education (2nd ed., Vol. 7: Language testing and assessment, pp. 257-571). New York: Springer.Scardamalia, M., & Bereiter, C. (1987). Knowledge telling and knowledge transforming in written composition. In S. Rosenberg (Ed.), Advances in applied psycholinguistics, Volume 2: Reading, writing, and language learning (pp. 142-175). Cambridge: Cambridge University Press.Skehan, P. (1989). Individual differences in second-language learning. London: Arnold.Skehan, P. (1996). A Framework for the implementation of task-based instruction. Applied Linguistics, 17(1), 38-62. doi: 10.1093/applin/17.1.38Skehan, P. (1998). A cognitive approach to language learning. Oxford: Oxford University Press.Skehan, P. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30(4), 510-532. doi: 10.1093/applin/amp047Storch, N. (2002). Relationships formed in dyadic interaction and opportunity for learning. International Journal of Educational Research, 37(3–4), 305-322. doi: 10.1016/s0883-0355(03)00007-7Storch, N. (2005). Collaborative writing: Product, process, and students’ reflections. Journal of Second Language Writing, 14(3), 153-173. doi: 10.1016/j.jslw.2005.05.002Storch, N. (2009). The impact of studying in a second language (L2) medium university on the development of L2 writing. Journal of Second Language Writing, 18(2), 103-118. doi: 10.1016/j.jslw.2009.02.003Storch, N., & Wigglesworth, G. (2007). Writing tasks: The effects of collaboration. In M. d. P. García Mayo (Ed.), Investigating tasks in formal language learning (pp. 157-177). Clevedon, England: Multilingual Matters.Sweedler-Brown, C. O. (1993). ESL essay evaluation: The influence of sentence-level and rhetorical features. Journal of Second Language Writing, 2(1), 3-17. doi: (93)90003-LTapia, E. (1993). Cognitive demands as a factor in interlanguage syntax: A study in topics and texts. (Unpublished doctoral dissertation). Indiana University, Bloomington. Retrieved from Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: SAGE.Turner, C. E., & Upshur, J. A. (2002). Rating scales derived from student samples: Effects of the scale maker and the student sample on scale content and student scores. TESOL Quarterly, 36(1), 49-70. doi: 10.2307/3588360Upshur, J. A., & Turner, C. E. (1999). Systematic effects in the rating of second-language speaking ability: Test method and learner discourse. Language Testing, 16(1), 82-111. doi: 10.1177/026553229901600105Wall, D., & Alderson, J. C. (1993). Examining washback: The Sri Lankan impact study. Language Testing, 10(1), 41-69. doi: 10.1177/026553229301000103Weigle, S. C. (2002). Assessing writing. Cambridge: Cambridge University Press.Weigle, S. C. (2012). Assessing writing. In C. Coombe, P. Davidson, B. O'Sullivan & S. Stoynoff (Eds.), The Cambridge guide to second language assessment (pp. 218-224). Cambridge: Cambridge University Press.White, E. M. (1984). Holisticism. College Composition and Communication, 35(4), 400-409. doi: 10.2307/357792White, E. M. (1995). An apologia for the timed impromptu essay test. College Composition and Communication, 46(1), 30-45. doi: 10.2307/358868Wigglesworth, G., & Storch, N. (2009). Pair versus individual writing: Effects on fluency, complexity and accuracy. Language Testing, 26(3), 445-466. doi: 10.1177/0265532209104670Wigglesworth, G., & Storch, N. (2012). What role for collaboration in writing and writing feedback. Journal of Second Language Writing, 21(4), 364-374. doi: , K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy & complexity. Honolulu, HI: University of Hawai'i, Second Language Teaching and Curriculum Center.Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Thousand Oaks, CA: SAGE.Zentralstelle für das Auslandsschulwesen. (2012). Deutsches Sprachdiplom der Kultusministerkonferenz: Handreichungen für die schrifltliche Kommunikation (Niveaustufe B2/C1). [German language diploma of the education ministers' conference: Documents for the written communication (Level B2/C1)] Retrieved from Table 1 Gender, Age, Degree Program, and L1 Background of Student ParticipantsStudents in Course 1Students in Course 2Gender10 female, 9 male7 female, 7 maleAge18-40 (M = 24.5, SD = 6.3)18-39 (M = 23.1, SD = 6.0)L1 Background 4 Chinese, 3 Arabic, 3 French, 2 Farsi, & 7 others5 Chinese, 3 Arabic, 2 French, & 4 othersDegree Program10 Finance, Business, & Accounting3 Engineering2 Social Science2 Science1 Languages & Literature1 Art History & Fine Arts6 Finance, Business, & Accounting2 Engineering2 Social Science2 Science1 Language & Literature1 Art History & Fine ArtsTable 2 Means and Standard Deviations for the Grammar Grade & Accuracy and Complexity Measures by CourseCourse 1Course 2MSDRangeMSDRangeErrors per 100 words6.843.551.65-12.524.62.21.75-8.72Clauses per T-unit2.360.351.84-3.002.130.301.47-2.28Words per Clause6.480.675.06-7.637.731.225.97-9.54Grammar Grade71.325.5363-8576.010.4160-86Table 3Correlations between the Grammar Grade and the Accuracy and Complexity Measures by CourseCourse 1Course 2Errors per 100 words-.433-.868**Clauses per T-unit.266-.532Words per Clause-.015.299* p < .05, ** p < .01Appendix AEvaluation Grid for Course 1(The evaluation grid for Course 2 has the same format, but the weighting of the criteria differs.)GUIDE FOR COMPLETING THE EVALUATION GRID (Course 1)Content: Ideas & Information (25%) Thesis statement (explicit, identifiable; appropriate to essay type or topic; predictive)Topic development (depth and quality/originality of information) Support (relevant, sufficient, detailed; general vs. specific support, fact vs. opinion)Information level/valueExcellent (A+, A, A-) Very clear and appropriate thesis, defined and supported with sound generalizations and substantial, specific, and relevant details; distinctive, original content for maximum impact; excellent information level; strong introduction and conclusion. (Very) Good (B+, B, B-) Clear and appropriate thesis; selects; suitable and appropriate content with sufficient details; informative; occasional minor problems with focus, depth, and/or unity; good introduction and conclusion. Satisfactory (C+, C, C-) Thesis may be unclear (e.g. too broad/narrow); acceptable topic development; some support points may be vague, insufficient, obvious, unconvincing; satisfactory introduction and conclusion.Weak (D+, D, D-) Thesis not apparent or weak; poor topic development; lacking in substance; many support points are insufficient, irrelevant and/or repetitive; low information level; weak conclusion. Fail (F) lacks main idea; unacceptable topic development; too vague, insufficient, unconvincing, or off-topic; not enough to evaluate. Organization & Text Structure (20%)Presence and logical sequencing of introduction, body paragraphs, and conclusionUse of relevant patterns of organization (related to topic or essay type)Coherent and unified relationship of ideas (NB: grammatical accuracy related to cohesive devices is considered under Grammar & Language Use)Excellent (A+, A, A-) - exceptionally clear plan connected to thesis; well organized, effective and logical sequencing; smooth flow of ideas; excellent use of transition techniques; clarity of message enhanced by organization.(Very) Good (B+, B, B-) - appropriate pattern of organization relevant to topic or essay type; generally smooth flow of ideas and appropriate use of transition techniques; overall organization good; most transitions used appropriately but would benefit from more frequent and varied use of transitions; sequencing generally logical.Satisfactory C+, C, C-) - shows understanding of pattern of development; somewhat choppy; relationships between ideas not always clear; overall organization satisfactory, but some elements may be loosely connected or lacking in transitions; most points logically sequenced but some problems in organization still exist.Weak (D+, D, D-) - problems with pattern of organization; disjointed; ideas do not flow well and relationships between ideas are often not clear; ideas difficult to follow because they are often not logically sequenced and/or are unrelatedFail (F) - does not show understanding of pattern of organization; no clear organization: confusing, vague, or seemingly unrelated ideas; pattern of organization not pertinent to topic/essay type; ideas not developed in separate paragraphs; not enough text to evaluateGrammar & Language Use (40%)Sentence structure (coordination and subordination; variety) Sentence problems (fragments, comma splices, run-ons)Verb structures (agreement, tense, form)Phrase structureArticles, pronouns, prepositionsExcellent (A+, A, A-) – sentences skillfully constructed, effectively varied with simple and complex forms; harmonious agreement of content and sentence design; hardly any errors in basic sentence or grammatical forms(Very) Good (B+, B, B-) – sentences accurately and coherently constructed with some variety; good use of complex constructions; only a few errors in grammatical forms; meaning not affected by errors.Satisfactory (C+, C, C-) - effective but simpler constructions and/or problems with complex constructions; meaning generally clear; several errors in grammatical forms.Weak (D, D+, D-) - some problems in simple constructions and/or frequent problems in complex constructions, or avoidance of complex structures; clarity weakened by awkward grammatical structures; many problems in grammatical forms.Fail (F) - many problems in sentence structures (both simple and complex) and/or absence of complex structures; frequent sentence structure errors which confuse and distract the reader; frequent errors in grammatical forms; not enough text to evaluate.Vocabulary (Terminology) (10%)Word formsWord choice (precision)RegisterIdiomatic usageRangeExcellent (A+, A, A-) high level of sophistication; impressive range; effective use of vocabulary to express ideas; only a few minor errors with word choice/form/idiom. (Very) Good (B+, B, B-) – (very) good range and variety in the use of vocabulary; effective word/idiom choice and usage; appropriate register; several minor errors related to word choice/form/idiom.Satisfactory (C+, C, C-) – adequate range in the use of vocabulary; occasional errors of word choice/form/idiom or usage, meaning generally clear (some minor ambiguity).Weak pass (D+, D, D-) - limited range; frequent errors of word choice/form/idiom and usage; meaning sometimes unclear or ambiguous as a result of errors.Fail (F) - very limited range; words recycled, reused, or too general; frequent errors of word choice/form/idiom and usage may obscure the meaning; problems with basic vocabulary; not enough text to evaluate.Mechanics (5%)Punctuation SpellingCapitalization Presentation (NB: punctuation involving fragments, comma splices and run-ons are considered underGrammar & Language Use)Excellent, (A+, A, A-) – very few errors either in punctuation, spelling, or capitalization; correct indentation; neat presentation.(Very) Good (B+, B, B-) - only a few minor errors in punctuation, spelling, and capitalization; clarity of message never affected by errors; correct indentation; legible handwriting.Satisfactory (C+, C, C-) - occasional errors in punctuation, spelling or capitalization, problems with indentation; meaning still clear despite errors; handwriting hard to read but basically legible.Weak (D+, D, D-) - many errors in punctuation, spelling, capitalization; meaning sometimes unclear as result of mechanical errors; absence of indentation; nearly illegible handwriting affecting text comprehensionFail (F) - dominated by errors in punctuation, spelling, indentation and capitalization; illegible handwriting.Appendix B:Student QuestionnaireInformation about youWhat is your age? (Please state) ____________What is your gender? (Please check) _____ female _____ maleWhich language did you FIRST learn to speak? (Please state)______________________________________________________________Which other languages do you speak? (Please state)______________________________________________________________How long have you lived in Canada? (Please check)_____ 1 year or less_____ 2-4 years_____ 5 years or longer_____ I was born in CanadaWhat is your current student status at Concordia University? (Please check)_____ Undergraduate student _____ MA student_____ PhD studentWhat is your major field of study? (Please state) ______________________________________________________________What is your current semester in your degree program at Concordia University? _____ 1st semester_____ 2nd semester_____ 3rd semester_____ 4th semester_____ 5th semester or higherIs this your first ESL course at Concordia University? _____ yes _____ noIf you answered NO, which courses have you already taken? (Please state)____________________________________________________________How did you learn English? (Please check ALL that apply)_____ English classes in my country. Please state country: _________________ English classes in Canada_____ With friends in Canada_____ At work in Canada_____ Other, please state: ______________________________________What is the total number of years you have been studying English since you started school? (Please state) ____________How often do you use English outside of classes? (Please check only ONE)_____ Never_____ Almost never_____ A few times a week_____ At least once every day _____ All the timeFor what purposes do you use English? (Please check ALL that apply)_____ At university_____ At work_____ With my friends_____ At home_____ Other, please state: ______________________________________Questions on the Assessment of GrammarWhen your teacher evaluates your essays, he/she uses an evaluation grid. One of the marks on the grid is for language use & grammar. The following questions all refer to this grammar mark on the evaluation grid.What is grammar to you?What do you think your teacher looks for in your essays when he/she assesses grammar?Do you understand why you receive a particular grade for grammar on your essays?_____ Yes _____ No _____ Don’t knowComments:What do you focus on when you’re writing an essay exam in English?Do you think about the grammar mark when you write an essay exam in English?_____ All the time_____ Sometimes_____ Not really_____ Don’t knowComments:Do teacher’s expectations for grammar affect the way you write? Explain why or why not.Appendix CProtocol for Student InterviewsWelcome and chat with the student.Thank you very much for taking the time today to come here for this interview. It is very important for me to hear what you have to say about the assessment of grammar. I really appreciated your responses on the questionnaire. I would like your help in understanding some of them in more depth.Before you agreed to participate in my study, had you ever thought about how teachers assess students’ work?Have you given some thought to what grammar represents or means to you? What do you think your teacher looks for in your essays when she assesses grammar?Have you thought about why you receive a particular grade for grammar on your essays?What do you focus on when you’re writing an essay exam in English?Do you think about the grammar mark when you write an essay exam in English?Do your teacher’s expectations for grammar affect the way you write? Explain why or why not.I have asked you a lot of questions and you’ve been very helpful. But do you think there’s anything we’ve missed out?Do you have any other comments about what we’ve discussed?Appendix DProtocol for Teacher InterviewsGreet and chat with the teacher.Thank you very much for taking the time today for this interview. I would like your help in understanding your expectations in terms of grammar performance…and your beliefs. It is very important for me to hear what you have to say about the assessment of grammar as it will allow me to understand the assessment process much better. When you sit down to assess your students’ essays, what is the biggest problem or challenge you face?What do you tell your students they should focus on when writing an essay?When assigning the grammar mark, what do you look for in an essay?What is the most important aspect for you when deciding which grammar mark to give?11049031877000Which criteria do you use to distinguish between the different grades levels A, B, C, etc.?-424019221615For each of the four essays00For each of the four essaysWhat are the characteristics of this particular essay that cause you to assign it Grade X for grammar? If the student who wrote this essay would ask you about why he/she got this particular mark on grammar, what would you tell him/her?When you discuss grammar marks for your students’ essays with your colleagues, what do you usually agree on?When you discuss grammar marks for your students’ essay with your colleagues, what do you usually disagree on?I have asked you a lot of questions and you’ve been very helpful. But do you think there’s anything we’ve missed out?Do you have any other comments about what we’ve discussed?Appendix EEssay Topics on the Midterm ExamTopics for Course 1Write an essay with examples illustrating the drawbacks of living gin the digital age, i.e., a time in which electronic devices are prevalent.There are different ways to escape the stress of modern life. Write an essay using specific examples to support his statement.There are several important life skills that people need in order to be successful in the workplace. Write an essay using specific examples to support this ics for Course 2We have all experienced stress in our lives. It comes from our work, our family life, and our relationship with people. Discuss the positive effects of stress in our lives.Fast-food restaurants are everywhere. They are in shopping malls, office buildings, city centers, and neighbourhoods. Discuss EITHER the causes OR the effects of the popularity of fast food restaurants.Despite the social safety net and various assistance programs run by governments and charity organizations, we see many homeless people in streets, in parks, and at metro stations. Discuss the causes of homelessness. ................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- listen a esl listening job interviews
- preval skill testing request form
- diagnostic grammar test ryerson university
- secondary key stage 3
- title teacher assessment of grammatical ability in second
- literacy and english experiences and outcomes
- grammar worksheet 1
- worksheet on chapter 9 exercise on pronoun case
- unit 8 grammar practice
Related searches
- another word for ability in resume
- teacher assessment tools examples
- title ix department of education
- assessment for learning vs assessment of learning
- examples of grammatical errors
- grammatical errors in writing
- title for founder of nonprofit
- 3 learning needs assessment template for nurses in pdf
- title 44 code of federal regulations
- examples of cognitive ability tests
- title 36 code of federal regulations
- assessment for learning and assessment of learning